Expect

Expect is a Unix automation tool and scripting language created by Don Libes at the National Institute of Standards and Technology (NIST) in 1990. Built as an extension of John Ousterhout’s Tcl language, Expect specializes in a problem that resisted ordinary shell scripting: automating interactive programs that expect input from a human at a terminal. By orchestrating a child process through a pseudo-terminal and pattern-matching its output, Expect made tools like telnet, ftp, passwd, ssh, and countless legacy applications scriptable for the first time.

History & Origins

Don Libes began work on Expect while at NIST in the late 1980s. He was frustrated by a recurring class of automation problems: many Unix utilities deliberately bypass standard input redirection and read directly from the terminal, making them impossible to drive from a shell script. Programs like passwd, telnet, and ftp checked whether they were attached to a real TTY and refused to cooperate with naive piping.

Libes solved this by allocating a pseudo-terminal (pty) for the child process, fooling the program into believing a human was on the other end. On top of that mechanism he layered a small set of Tcl commands - principally spawn, send, expect, and interact - that allowed scripts to wait for output patterns and respond accordingly.

The 1990 USENIX paper “Expect: Curing Those Uncontrollable Fits of Interaction” introduced the tool to the broader Unix community and remains widely cited. Expect was recognized with a Federal 100 Award in the early 1990s and the USENIX STUG (Software Tools User Group) Award in 1994, recognizing its impact on government and academic computing.

The Name

The name “Expect” comes from the central command of the language: expect, which waits for a particular pattern to appear in the output of the spawned process. A typical script reads almost like a dialogue: spawn a program, expect a prompt, send a response, expect the next prompt, and so on.

Design Philosophy

Expect is fundamentally a conversational language. A script is essentially a transcript of an expected interaction, with branching for the cases where things deviate. Several principles shape it:

1. Programs are dialogue partners - Every spawned process is treated as something you converse with via patterns and responses
2. Pattern matching is central - The expect command supports glob patterns, regular expressions, and exact matches
3. Inherit from Tcl - Rather than invent new syntax, Expect reuses Tcl’s procedures, control flow, and string handling
4. Pseudo-terminal transparency - The pty layer is invisible to the script author; programs simply behave as they would for a human user
5. Timeouts are first-class - Every wait is bounded, because real interactive programs sometimes hang

Key Features

The Core Quartet

Most Expect scripts are built from four commands:

1
2
3
4

spawn ssh user@host         ;# Start a child process under a pty
expect "password:"          ;# Wait for this pattern in output
send "secret\r"             ;# Send input to the child
interact                    ;# Hand control back to the user

Pattern Matching with Multiple Branches

expect can wait for several patterns at once and dispatch on whichever appears first:

1
2
3
4
5
6

expect {
    "Permission denied" { puts "auth failed"; exit 1 }
    "Last login"        { puts "logged in" }
    timeout             { puts "host unreachable"; exit 2 }
    eof                 { puts "connection closed"; exit 3 }
}

This makes Expect well-suited for scripting against systems where many outcomes are possible.

Timeouts and EOF

Every expect operation has an associated timeout (default 10 seconds, configurable via the timeout variable) and implicitly handles end-of-file. This guards scripts against hanging when a remote system goes silent.

Interact Mode

The interact command transfers control between the human running the script and the spawned process, with the option to install local “macros” that intercept keystrokes. This enables hybrid scripts where automation handles the boring parts and a human takes over for judgment calls.

Autoexpect

Expect ships with autoexpect, a tool that records a live interactive session and emits an Expect script that reproduces it. This lowers the barrier to writing scripts for unfamiliar interactive programs.

A Typical Expect Script

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

#!/usr/bin/env expect

set timeout 30
set host    [lindex $argv 0]
set user    [lindex $argv 1]
set passwd  [lindex $argv 2]

spawn ssh $user@$host
expect {
    "yes/no"     { send "yes\r"; exp_continue }
    "password:"  { send "$passwd\r" }
    timeout      { puts "timed out"; exit 1 }
}
expect "$ "
send "uptime\r"
expect "$ "
send "exit\r"
expect eof

The script connects to a host, handles the first-time host-key prompt if present, sends a password, runs uptime, and disconnects.

Evolution

Expect’s evolution has been one of stability rather than reinvention. The 5.x series, which has been current for more than two decades, refined performance, debugging, and platform support without altering the core programming model. The maintenance pace slowed considerably in the 2010s, with Expect 5.45.4 (2018) being the most recent widely packaged release at the time of writing.

Rather than a new Expect, the community produced ports of the Expect concept into other ecosystems:

These derivatives carry the spawn/expect/send model into languages that are more familiar to modern developers, while the original Tcl-based Expect remains the canonical reference.

Current Relevance

Expect is no longer a growth area, but it is far from dead. It survives in domains where other tools are awkward:

• Network operations - Many production network automation systems still rely on Expect or its descendants for devices that lack proper APIs
• Compiler and debugger testing - DejaGnu, the GNU testing framework, drives GCC, GDB, and Binutils test suites with Expect scripts
• Sysadmin glue - One-off Expect snippets remain a common way to automate passwd, ssh-keygen, and other interactive utilities
• Legacy and embedded systems - Industrial, telecom, and government systems whose only management interface is a terminal continue to depend on Expect-style automation

The Python-based Pexpect now sees considerably more new development than upstream Expect, but scripts written against the original Tcl-based tool from the 1990s and 2000s frequently still run unchanged today.

Why It Matters

Expect’s enduring importance lies less in its specific syntax and more in the idea it operationalized: that interactive programs can be controlled by other programs, given a pseudo-terminal and a willingness to pattern-match output. Before Expect, “interactive” effectively meant “not automatable.” After Expect, the divide between batch tools and interactive tools became negotiable.

That conceptual shift has shown up everywhere from continuous integration systems to network automation platforms to AI agents that drive command-line tools. Every modern library that “drives” a CLI by reading and responding to its output is, in a real sense, doing what Don Libes proposed in 1990.

Getting Started

Expect scripts typically use the .exp extension and start with a shebang pointing at the expect interpreter:

1

#!/usr/bin/env expect

A typical workflow:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# Install (Debian/Ubuntu)
sudo apt-get install expect

# Install (macOS via Homebrew)
brew install expect

# Run a script
expect myscript.exp

# Or make it executable
chmod +x myscript.exp
./myscript.exp

Because Expect is built on Tcl, full familiarity with Tcl is helpful: Tcl provides the variables, control flow, procedures, and string handling, while Expect adds the process-control and pattern-matching commands on top.