Lesson 3: PTY & Terminal Driver

The pseudoterminal (PTY) is the kernel abstraction that makes terminal emulators possible. It is the bridge between a GUI application (Ghostty) and the command-line programs running inside it. After this lesson, you understand the full byte path from a keystroke in Ghostty to a character appearing in vim.

PTY Architecture #

A PTY is a pair of character devices:

flowchart LR
    subgraph Emulator["Terminal Emulator"]
        G["Ghostty<br/>reads/writes master fd"]
    end
    subgraph Kernel["Kernel"]
        M["master fd<br/>/dev/ptmx"]
        LD["Line Discipline"]
        S["slave fd<br/>/dev/pts/N"]
    end
    subgraph Child["Child Process"]
        Z["zsh / vim<br/>fds 0,1,2"]
    end
    G <-->|"read/write"| M
    M --- LD
    LD --- S
    S <-->|"stdin/stdout/stderr"| Z

• Master fd (/dev/ptmx): the emulator's end. Writing to master sends bytes to the slave (like typing). Reading from master receives bytes the slave wrote (output to display).
• Slave fd (/dev/pts/N): the child process's end. Writing to slave sends data to the master (program output). Reading from slave receives data written to master (user input).

Creating a PTY #

#define _XOPEN_SOURCE 600
#include <stdlib.h>
#include <fcntl.h>

int master_fd, slave_fd;
pid_t pid;

master_fd = posix_openpt(O_RDWR | O_NOCTTY);
grantpt(master_fd);
unlockpt(master_fd);
char *slave_name = ptsname(master_fd);

pid = fork();
if (pid == 0) {
    // Child
    setsid();  // new session, no controlling terminal
    slave_fd = open(slave_name, O_RDWR);
    close(master_fd);
    dup2(slave_fd, STDIN_FILENO);
    dup2(slave_fd, STDOUT_FILENO);
    dup2(slave_fd, STDERR_FILENO);
    close(slave_fd);
    execlp("zsh", "zsh", NULL);
}
// Parent: read/write master_fd to communicate with child

Key steps:

1. posix_openpt() opens the master, returns master fd
2. grantpt() / unlockpt() set permissions on the slave
3. fork() creates the child
4. setsid() creates a new session (required for the slave to become the controlling terminal)
5. Child opens the slave and dups it to 0/1/2
6. Parent communicates via master fd

The Line Discipline #

The line discipline is a kernel module that sits between the master and slave ends of the PTY. It processes the byte stream before the child process sees it.

graph LR
    A[Master fd] --> B[Line Discipline]
    B --> C[Slave fd]
    C --> B
    B --> A

What the line discipline does:

1. Buffering and line editing (canonical mode): in cooked mode, the line discipline buffers characters until Enter is pressed. It handles backspace (erase previous character), Ctrl+W (erase word), Ctrl+U (erase line). The program only receives complete lines.
2. Echo: in cooked mode, characters typed are echoed back to the master (so the terminal emulator can display them). In raw mode, echo is disabled — the program handles display.
3. Signal generation: specific control characters generate signals: Ctrl+C → SIGINT (VINTR), Ctrl+\ → SIGQUIT (VQUIT), Ctrl+Z → SIGTSTP (VSUSP).
4. Flow control: Ctrl+S (XOFF) pauses output. Ctrl+Q (XON) resumes. The terminal driver inserts XOFF when the buffer is nearly full.
5. Carriage return processing: \n → \r\n conversion (ONLCR).

termios Deep Dive #

The termios structure controls the line discipline's behavior:

struct termios {
    tcflag_t c_iflag;   // input flags
    tcflag_t c_oflag;   // output flags
    tcflag_t c_cflag;   // control flags
    tcflag_t c_lflag;   // local flags (the important ones)
    cc_t     c_cc[NCCS]; // special control characters
};

The lflag (local flags) — what you actually care about #

Raw mode setup (what Ghostty does) #

struct termios raw;
tcgetattr(slave_fd, &raw);
cfmakeraw(&raw);  // clears everything useful
// Or manually:
raw.c_lflag &= ~(ECHO | ICANON | ISIG | IEXTEN);
raw.c_iflag &= ~(IXON | ICRNL | BRKINT | INPCK | ISTRIP);
raw.c_oflag &= ~(OPOST);
raw.c_cflag |= (CS8);
raw.c_cc[VMIN] = 1;    // read() returns after 1 byte
raw.c_cc[VTIME] = 0;   // no timeout
tcsetattr(slave_fd, TCSAFLUSH, &raw);

In raw mode:

• No line buffering (ICANON off): read() returns immediately for each byte
• No echo (ECHO off): the program decides what to display
• No signal generation (ISIG off): Ctrl+C is just a byte (0x03), not SIGINT
• No flow control (IXON off): Ctrl+S/Q are just bytes
• No output processing (OPOST off): \n stays \n, no \r\n conversion

Process Groups and Sessions #

Terminal job control depends on three kernel concepts:

Session #

A session is a collection of process groups. Each session has at most one controlling terminal. setsid() creates a new session and makes the calling process the session leader.

Process group #

A process group is a set of processes that should receive signals together. setpgid(pid, pgid) sets the process group. The foreground process group receives terminal-generated signals (SIGINT, SIGTSTP). Background process groups receive SIGTTIN/SIGTTOU if they try to read/write the terminal.

Controlling terminal #

Each session can have one controlling terminal (the slave side of a PTY). tcsetpgrp(fd, pgid) sets which process group is in the foreground. The foreground group receives keyboard signals. Only the foreground group can read from the terminal.

graph TD
    subgraph Session
        A[Session Leader<br>shell] --> B[Foreground Group<br>pgid=100<br>vim]
        A --> C[Background Group<br>pgid=200<br>sleep 100 &]
    end
    A -.-> D[Controlling Terminal<br>/dev/pts/5]

Job Control Mechanics #

Ctrl+Z (suspend) #

1. User presses Ctrl+Z
2. Line discipline receives 0x1A (ASCII SUB)
3. If ISIG is set, line discipline sends SIGTSTP to the foreground process group
4. Default action: suspend all processes in the group (state changes to TASK_STOPPED)
5. Parent (shell) receives SIGCHLD, discovers child stopped (WIFSTOPPED)
6. Shell puts the job in background, updates its job list
7. Shell calls tcsetpgrp() to reclaim terminal control

fg command #

1. Shell calls tcsetpgrp(shell_terminal, job_pgid) — gives terminal to the job
2. Shell calls kill(-job_pgid, SIGCONT) — resumes the stopped processes
3. Shell calls waitpid() — blocks until the job finishes or stops again

bg command #

1. Shell calls kill(-job_pgid, SIGCONT) — resumes the stopped processes
2. Shell does NOT wait — the job runs in the background
3. If the job tries to read from terminal: kernel sends SIGTTIN, which stops it again

Window Size #

struct winsize {
    unsigned short ws_row;     // rows in characters
    unsigned short ws_col;     // columns in characters
    unsigned short ws_xpixel;  // width in pixels
    unsigned short ws_ypixel;  // height in pixels
};

ioctl(master_fd, TIOCSWINSZ, &ws);  // set size
ioctl(master_fd, TIOCGWINSZ, &ws);  // get size

When the emulator window resizes:

1. Ghostty calculates new rows/cols based on font metrics and pixel dimensions
2. Calls ioctl(master_fd, TIOCSWINSZ, &new_size)
3. Kernel sends SIGWINCH to the foreground process group
4. Applications (vim, tmux, less) catch SIGWINCH and re-render

ptycat Project #

Build a PTY inspector: open a PTY, spawn a shell inside, and hex-dump the byte stream flowing through the master fd. This is your first look at the raw escape sequences that terminal programs emit.

// ptycat: open pty, fork shell, print hexdump of master reads
master_fd = posix_openpt(O_RDWR | O_NOCTTY);
grantpt(master_fd);
unlockpt(master_fd);

pid = fork();
if (pid == 0) {
    setsid();
    slave_fd = open(ptsname(master_fd), O_RDWR);
    close(master_fd);
    dup2(slave_fd, 0); dup2(slave_fd, 1); dup2(slave_fd, 2);
    close(slave_fd);
    execlp("zsh", "zsh", NULL);
}

// Set raw mode on the slave (important!)
// Read from master_fd, hexdump each byte, write user input to master_fd
while (1) {
    char buf[4096];
    int n = read(master_fd, buf, sizeof(buf));
    for (int i = 0; i < n; i++) {
        printf("%02x ", (unsigned char)buf[i]);
        if (i % 16 == 15) printf("\n");
    }
}

Run it, type some commands, and watch the escape sequences fly. Then run vim and watch the bytes explode.

Ghostty Source to Study #

Self-Check #

Can you:

• Draw the full PTY architecture (master, slave, line discipline, child process)
• Write the openpty() + fork() + dup2() + exec() sequence from memory
• Explain the difference between cooked mode and raw mode — name at least 5 flags that differ
• Explain why VMIN=1, VTIME=0 matters for a terminal emulator
• Trace the full pathway of Ctrl+Z: keystroke → line discipline → SIGTSTP → process group → shell notification
• Explain why setsid() is necessary when creating a PTY child process
• Write code that responds to window resize and propagates TIOCSWINSZ to the slave