Technology1 min read

We can do better than line-buffered output

By · Published by Everything Blog

In short

Tools like Info-ZIP unzip demonstrate how terminal line buffering can drastically inflate execution times, often forcing users to redirect output or use quiet modes. System tracing shows that standard environments like glibc and CPython repeatedly initiate costly, small writes during sequential line production. Replacing this behavior with an idle-flush system inspired by Nagle's algorithm relies on a perception-based deadline of around 100 milliseconds. The tugz extraction tool successfully deploys this technique between tasks to match fully buffered speeds while maintaining prompt terminal displays.

Key points

  • Line-buffered terminal output creates excessive small write calls that heavily penalize s…: Line-buffered terminal output creates excessive small write calls that heavily penalize sequential text-generating commands.
  • An idle-flush method uses a roughly 100-millisecond deadline to aggregate output without…: An idle-flush method uses a roughly 100-millisecond deadline to aggregate output without human-perceptible display delays.
  • Programs like the tugz unzip utility achieve fully buffered performance by checking idle…: Programs like the tugz unzip utility achieve fully buffered performance by checking idle deadlines between short periods of work.

Sources checked by our analysis pipeline: 1

nullprogram.com/blog/2026/11/05/

Buffered output typically comes in two flavors: fully buffered

and line buffered. If output goes to a terminal then the implementation

automatically selects line buffering, flushing its buffer when a line is

completed so that it reaches the user immediately. Otherwise it fully

buffers, holding output until the buffer is full or an explicit flush

event at some arbitrary point in the future. Line buffering is a common

bottleneck in non-interactive terminal applications such that it’s worth

redirecting output to a file or pipe in order to avoid it. It doesn’t

have to be this way. We can have our cake and eat it, too: promptly

delivering lines to a terminal at fully-buffered speeds.

For example, Info-ZIP unzip

prints file names as it extract them from an

archive. This printout dominates extraction time of archives with many

small files. Sometimes I run unzip

without options, then my terminal

floods with file names and I know that my extraction is bottlenecked. So I

cancel and restart with -q

(quiet) to avoid the bottleneck, because it’s

faster to start over than…

Technology