Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool

blog.robertelder.org

Post reply on HN
Print view View original

41–50 of 106 posts

Re: Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool

#41

Post by lilyball »

robertelder wrote: post #37
dekhn wrote: post #35

Earlier quoted context omitted.

Sure, let's say you want to remotely control a Cisco router with a serial interface. expect makes it really easy to remotely connect to the serial interface, wait for a prompt, send a command, and take a conditional action that depends on the response.

I was thinking more of code examples. Expect is capable of doing much more than just the basic 'expect', then 'send' pattern. For example, this article: 'Most programmers don't know what Expect can do for them' http://wiki.tcl.tk/3913

The article mentioned it's used for testing gcc and LLVM. If you want an example you can actually look at, it's also used for testing the fish-shell (fish-shell has other tests too, but expect is specifically used to actually test user-visible behavior, which makes expect the perfect tool for the task). You can find those tests in https://github.com/fish-shell/fish-shell/tree/master/tests. The files of the form "foo.expect" are the tests (the "foo.expect.err" and "foo.expect.out" files are how it interacts with the rest of the testing infrastructure), and the file interactive.expect.rc provides a bunch of helper functions for the expect scripts, and in particular a standard way of looking for prompts (because the shell will redraw the prompt at various times, this function, coupled with the fish code in interactive.config, lets the expect scripts tell when it's the old prompt redrawn versus the new prompt after a command has been executed).

Re: Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool

#42

Post by lamontcg »

seanp2k2 wrote: post #38
wyldfire wrote: post #7

Earlier quoted context omitted.

I 100% agree. If I can use paramiko/fabric or something similar, I prefer it way more than expect. If you have a slave device over RS232, expect is probably your only option. But if that remote item is running sshd, you are way better off without expect.

I've used paramiko + expect to automate distributed benchmarking + results gathering. I can say that while it ultimately worked, Curse That Thing In Particular. It was so so so fiddly to get everything working, and very fragile + output-dependent. https://github.com/ip2k/Mad-Science?files=1 I wrote this a long time ago and it's ugly. It totally worked and let our company find more optimal hosting configs.

> while it ultimately worked, Curse That Thing In Particular

"Things Only 90s SysAdmins Will Remember"

Expect is mostly a tool of last resort when a utility has silly prompts that you can't bypass by hitting it with command line options like `-y` and stuff.

Re: Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool

#43

Post by rodgerd »

skywhopper wrote: post #19

Expect is a last ditch tool that becomes less and less necessary as automation libraries and remote APIs become more and more common. The examples given are cute but hardly representative of what expect is really for. The only times I've needed to use expect was to drive weird non-scriptable telnet-based remote systems and to automate unpackaging of intentionally-broken vendor-supplied self-expanding software archive…

Early versions of the local library web site we expect driving the AS/400 (IIRC) terminals on behalf of web interaction.

Re: Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool

#44

Post by znpy »

Speaking of surprisingly unappreciated unix tools, I recently started reading the documentation of the gnu info package, and I must say, that is really impressive.

if you didn't already, I highly recommend you type 'info info' in your terminal and spend half an hour learning how to use it.

Plus, if you are an emacs user, the emacs manual is bundled with emacs in info format, and you can obviously read info documents in emacs too.

Reading the gnu emacs manual inside emacs is a game-changer.

Re: Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool

#45

Post by sitkack »

rodgerd wrote: post #43
skywhopper wrote: post #19

Expect is a last ditch tool that becomes less and less necessary as automation libraries and remote APIs become more and more common. The examples given are cute but hardly representative of what expect is really for. The only times I've needed to use expect was to drive weird non-scriptable telnet-based remote systems and to automate unpackaging of intentionally-broken vendor-supplied self-expanding software archive…

Early versions of the local library web site we expect driving the AS/400 (IIRC) terminals on behalf of web interaction.

I can _hear_ the expect code running in lots of government IVR (interactive voice response) applications. Green scraping.

Re: Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool

#46

Post by seiferteric »

Karunamon wrote: post #16
jstimpfle wrote: post #15

Earlier quoted context omitted.

I think expect is often used to have it enter username and password "interactively". But it's easier and more robust to just use the public-key auth facility that is built-in to ssh.

Not always possible. The problem is that ssh lives in the mythical land of perfect security, where people never need to do unsafe things in order to get their jobs done. Hence keys silently not working without "correct" permissions, passwords not passable on the CLI, inconsistent type-yes-to-override behavior, and other user-hostile design choices. I love openssh, but it's a real pain in the ass sometimes.

there is also sshpass for supplying password, as well as using a languages library like paramiko for python which I believe allows you to supply a password. It would take more than password auth to force me to use expect these days :)

Re: Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool

#48

Post by hydandata »

Expect can be a lifesaver when you cannot use your usual go-to tools like Ansible etc. especially with legacy setups where you cannot tweak stuff much. I have successfully used it combined with tmux for automated multi-term like functionality, deployment verification and other chores to save a lot of $ for my company. I do try to keep my Expect scripts as simple as possible though, just so I do not have to remember too much since I do not work with TCL anywhere else.

Re: Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool

#49

Post by ChuckMcM »

This tool keeps the command line interpreter (CLI) alive. Sysadmins everywhere can glue hundred different systems from a hundred different vendors together if they have CLI and the sysadmin has access to a machine that runs expect(1).

There are many stories of complex infrastructure being taken down by new firmware on routers or switches that change the order of parameters in a command :-)

Re: Don Libes' Expect: A Surprisingly Underappreciated Unix Automation Tool

#50

Post by andlrc »

znpy wrote: post #44

Speaking of surprisingly unappreciated unix tools, I recently started reading the documentation of the gnu info package, and I must say, that is really impressive. if you didn't already, I highly recommend you type 'info info' in your terminal and spend half an hour learning how to use it. Plus, if you are an emacs user, the emacs manual is bundled with emacs in info format, and you can obviously read info documents…

`info` also contains a lot of historical notes, i.e.

    $ info Groff | awk '$2 == "History"{f = 4};f && f--' RS= ORS='\n\n'
    1.2 History
    ===========
    
    'troff' can trace its origins back to a formatting program called
    'RUNOFF', written by Jerry Saltzer, which ran on the CTSS (_Compatible
    Time Sharing System_, a project of MIT, the Massachusetts Institute of
    Technology) in the mid-sixties.(1)  (*note History-Footnote-1::) The
    name came from the use of the phrase "run off a document", meaning to
    print it out.  Bob Morris ported it to the 635 architecture and called
    the program 'roff' (an abbreviation of 'runoff').  It was rewritten as
    'rf' for the PDP-7 (before having UNIX), and at the same time (1969),
    Doug McIllroy rewrote an extended and simplified version of 'roff' in
    the BCPL programming language.
    
       In 1971, the UNIX developers wanted to get a PDP-11, and to justify
    the cost, proposed the development of a document formatting system for
    the AT&T patents division.  This first formatting program was a
    reimplementation of McIllroy's 'roff', written by J. F. Ossanna.
Post reply on HN