Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Non-interactive SSH password authentication

vincent.bernat.ch

Post reply on HN
Print view View original

111–120 of 128 posts

Re: Non-interactive SSH password authentication

#111

Post by latchkey »

xorcist wrote: post #104
latchkey wrote: post #75

Earlier quoted context omitted.

It isnt. I ended up building a small golang binary that ran as root and I could hit it with http calls to execute whatever I wanted. Built a message queue that would work through all the machines for eventual consistency. Worked great.

I've used both puppet and ansible at that scale and it's not only workable, it is a good idea.

How did you deal with machines randomly crashing and ensuring everything was consistent at all times across multiple physical datacenters?

I was also solving more problems than just ansible/puppet solve... like monitoring of specific systems on the machines.

Regardless, more than one way to skin a cat.

Re: Non-interactive SSH password authentication

#112

Post by generalizations »

IshKebab wrote: post #85
generalizations wrote: post #66

Earlier quoted context omitted.

On the other hand, text-based line-oriented APIs are a force multiplier that lets one solo dev/sysadmin be capable of administering (troubleshooting and building out and keeping alive) far more infrastructure than can be reasonably expected. And it's not sysvinit you should critique, but runit (fast and bulletproof - two things the other modern alternative doesn't really do). > Ignoring failures because they're moder…

I dunno I think something like JSON would still allow solo dev/sysadmins to get a lot done without the risk and effort of having to hand-roll separate parsers for every API. Then you could also offer something like Fuchsia's FIDL for programs to interface with - it allows generating the interface code fully typed in whatever language you want. No need to hand-roll a parser at all! > runit First I've heard of that. Lo…

I'm not familiar with Fuschia's FIDL system, but it looks intriguing. Though it's still pretty hard to beat using pipes and cut to ingest a line-oriented stream of text. If Fuchsia had beaten out UNIX in some parallel universe, it does look like we'd all be happier. Ha.

Runit is pretty good - it's the default in the Void Linux OS that I use on my personal machines. Definitely good enough for a modern desktop system, though it's not aimed at the same crowd as Ubuntu and Fedora.

Re: Non-interactive SSH password authentication

#113

Post by IshKebab »

generalizations wrote: post #112
IshKebab wrote: post #85

Earlier quoted context omitted.

I dunno I think something like JSON would still allow solo dev/sysadmins to get a lot done without the risk and effort of having to hand-roll separate parsers for every API. Then you could also offer something like Fuchsia's FIDL for programs to interface with - it allows generating the interface code fully typed in whatever language you want. No need to hand-roll a parser at all! > runit First I've heard of that. Lo…

I'm not familiar with Fuschia's FIDL system, but it looks intriguing. Though it's still pretty hard to beat using pipes and cut to ingest a line-oriented stream of text. If Fuchsia had beaten out UNIX in some parallel universe, it does look like we'd all be happier. Ha. Runit is pretty good - it's the default in the Void Linux OS that I use on my personal machines. Definitely good enough for a modern desktop system,…

> Though it's still pretty hard to beat using pipes and cut to ingest a line-oriented stream of text.

Well, a) that's fine for interactive use but awful for unattended use because it's so likely to go wrong, and b) it is actually pretty easy to beat that - JSON and jq is much easier, nicer and more reliable. Or Nushell/Powershell structured pipes. Or an autogenerated properly typed Python interface. Take a look at /proc/self/status before you tell me you'd rather use some awk/cut monstrosity than `jq .ppid`.

> If Fuchsia had beaten out UNIX in some parallel universe, it does look like we'd all be happier.

I'd definitely be happier - I wouldn't spend so much time debugging why Linux stuff breaks!

Re: Non-interactive SSH password authentication

#114

Post by linuxdude314 »

brewedDoritos wrote: post #110
xorcist wrote: post #103

Earlier quoted context omitted.

Read the man page instead of spending time on text generating tools. There is a batch mode in sftp which you should use. And sshpass will absolutely solve the lack of a missing tty in sftp. (Missing TERM is only a problem when you have a real tty, which you haven't.) Then again, don't do this. Use rsync. It is more robust and will avoid other problems in the future. And don't accept that a vendor only supports passwo…

Point taken about rsync, but you are assuming incorrectly that I haven't read it and that I haven't tried batch mode, but it still failed to run under cron inside an expect script. It's not my orgunit that manages the contract with the vendor. I will absolutely use paramiko instead of trying to explain a technical problem to two middle managers in another orgunit to take it up with a non technical person on the vendo…

As painful as what you have described may seem, it is the correct path.

Workarounds are fine as long as what you’re working around is documented and the business is aware.

Without at least trying to raise the issue, you are being “that guy”.

Re: Non-interactive SSH password authentication

#115

Post by downrightmike »

alsetmusic wrote: post #9

Wait, why aren't we using SSH keys? I just did a search on the page for 'key' and didn't see an explanation for why that's not the best option.

Bad(ancient) design, can you honestly say you look at that little image it generates and verify it is correct? Or actually read the acceptance of the key? No one reads that stuff or checks it, they just press yes to get to what they want to do.

Re: Non-interactive SSH password authentication

#116

Post by xorcist »

latchkey wrote: post #111
xorcist wrote: post #104

Earlier quoted context omitted.

I've used both puppet and ansible at that scale and it's not only workable, it is a good idea.

How did you deal with machines randomly crashing and ensuring everything was consistent at all times across multiple physical datacenters? I was also solving more problems than just ansible/puppet solve... like monitoring of specific systems on the machines. Regardless, more than one way to skin a cat.

That's the whole idea of using a system based on declarative state. As soon as the system is back up, the agent can resolve state again. You also keep a central copy of the state of every agent.

You can absolutely do this by writing your own agent (or by writing a family of bash script, but they tend to grow pretty complex over time), ansible is just a framework to write that in a standardized way. It will also out of the box handle a number of common system state such as running services and sysctl triggers.

There are a number of similar systems such as puppet or salt, which are all variations of the same basic idea. 30k hosts are a lot, and will need sizing the system appropriately, but it's not an unusual configuration by any means.

Re: Non-interactive SSH password authentication

#117

Post by latchkey »

xorcist wrote: post #116
latchkey wrote: post #111

Earlier quoted context omitted.

How did you deal with machines randomly crashing and ensuring everything was consistent at all times across multiple physical datacenters? I was also solving more problems than just ansible/puppet solve... like monitoring of specific systems on the machines. Regardless, more than one way to skin a cat.

That's the whole idea of using a system based on declarative state. As soon as the system is back up, the agent can resolve state again. You also keep a central copy of the state of every agent. You can absolutely do this by writing your own agent (or by writing a family of bash script, but they tend to grow pretty complex over time), ansible is just a framework to write that in a standardized way. It will also out o…

That was the benefit of my system over what you are talking about, there was no dependency on a global state or centralized control surface. Each worker was autonomous and self contained and had enough intelligence to bring itself to the desired state on its own. All you had to do was one line curl|bash install my service and it would take care of the rest without any other external dependencies. No worries about having to have ansible try to connect over and over again until things were working. It would just magically fix itself.

Again, many ways to skin the cat, but at the end of the day, this solution really worked extremely well. I would do it again in a heartbeat.

Re: Non-interactive SSH password authentication

#118

Post by qweqwe14 »

IshKebab wrote: post #108
qweqwe14 wrote: post #107

Earlier quoted context omitted.

1. The real fallacy here is assuming "it requires experience so it's impossible to do". Everything described in that article is trivial and anyone with experience in writing shell scripts knows about these things. People just instinctively go "this doesn't do what I expect therefore it's bad" instead of trying to understand the reasoning behind that. tl;dr Classic skill issue 2. Another fallacy. Obviously there will…

> anyone with experience in writing shell scripts knows about these things Who wants to have to be an expert in shell scripts to have to write them? In any case you're still wrong. Experts aren't immune to footguns. Post a complex shell script you've written. Let's see. > Obviously there will be badly written programs that don't handle paths properly. Like GNU Make? > It's absolutely trivial to do Trivial to do so th…

> Who wants to have to be an expert in shell scripts to have to write them?

If you aren't good with shell scripts, why bother? Go use Python or something else entirely. And it's not like you would use shell scripts to write all your programs, they aren't suitable for anything more complex than stiching a bunch of programs together anyway, yet people still try. And then those people complain that shell script is full of "footguns" when they are misusing it.

> Trivial to do so that it works for you. Absolutely not trivial to do so that it always works.

No idea what you are on about, literally every single detail you need to correctly parse /proc entries is in proc manpage. And yes, if you wrote your parser correctly it will still work between kernel versions, because /proc is considered a stable API between the kernel and userspace (same as syscalls). There are no "footguns" and no mysteries to it. It's a solved problem. Provide a concrete example of a footgun if there is one, and by "footgun" I mean something that isn't clearly stated in the manual that everyone dealing with /proc reads (right?)

Re: Non-interactive SSH password authentication

#119

Post by IshKebab »

qweqwe14 wrote: post #118
IshKebab wrote: post #108

Earlier quoted context omitted.

> anyone with experience in writing shell scripts knows about these things Who wants to have to be an expert in shell scripts to have to write them? In any case you're still wrong. Experts aren't immune to footguns. Post a complex shell script you've written. Let's see. > Obviously there will be badly written programs that don't handle paths properly. Like GNU Make? > It's absolutely trivial to do Trivial to do so th…

> Who wants to have to be an expert in shell scripts to have to write them? If you aren't good with shell scripts, why bother? Go use Python or something else entirely. And it's not like you would use shell scripts to write all your programs, they aren't suitable for anything more complex than stiching a bunch of programs together anyway, yet people still try. And then those people complain that shell script is full…

You have a fundamental misunderstanding of footguns, and humans. Explaining them is in manual doesn't mean they no longer exist. People often don't read the manual. Nor should they have to.

> If you aren't good with shell scripts, why bother? Go use Python or something else entirely

I agree! And given that essentially nobody is "good with shell scripts" we can simplify that advice to "don't write shell scripts".

Re: Non-interactive SSH password authentication

#120

Post by qweqwe14 »

IshKebab wrote: post #119
qweqwe14 wrote: post #118

Earlier quoted context omitted.

> Who wants to have to be an expert in shell scripts to have to write them? If you aren't good with shell scripts, why bother? Go use Python or something else entirely. And it's not like you would use shell scripts to write all your programs, they aren't suitable for anything more complex than stiching a bunch of programs together anyway, yet people still try. And then those people complain that shell script is full…

You have a fundamental misunderstanding of footguns, and humans. Explaining them is in manual doesn't mean they no longer exist. People often don't read the manual. Nor should they have to. > If you aren't good with shell scripts, why bother? Go use Python or something else entirely I agree! And given that essentially nobody is "good with shell scripts" we can simplify that advice to "don't write shell scripts".

> People often don't read the manual. Nor should they have to.

That's some kind of logic right there. If people don't read the manual, they have every right to expect that things will break. It's not surprising or shocking, it's just user error.

> we can simplify that advice to "don't write shell scripts".

No, we cannot simplify it like that. Shell scripts are a brilliant tool for a certain kind of problem, and they work well when used for that class of problem. When used for anything else, they work poorly. Same applies for literally any other tool. It's like saying "essentially nobody is good with a CNC machine so don't use it"

Post reply on HN