readline() no longer detects appended data since 80c1f1e45e8ef8c27d170fae7ade41971fe20218

[ntyni]

This is a bug report for perl from [email protected],
generated with the help of perlbug 1.43 running under perl 5.39.0.

---

As reported by Eric Wong in https://bugs.debian.org/1040947 it looks like
readline() no longer detects appended data if it gets eof on an empty
file first. Not sure if there are other conditions that trigger the issue.

This broke with commit 80c1f1e in 5.37.4.
(Note that we have backported the commit to Debian 5.36 packages so the Debian report concerns 5.36.0.)

I'm attaching Eric's original test script and my reduced one.

first bad commit: [80c1f1e] only clear the stream error state in readline() for glob()

tail.txt
readline.txt

---

Flags

• category=core
• severity=medium

---

Perl configuration

Site configuration information for perl 5.39.0:

Configured by ntyni at Sun Jul 16 08:12:30 UTC 2023.

Summary of my perl5 (revision 5 version 39 subversion 0) configuration:
  Commit id: 0c77b7235984453280b85b181aecb8e757ee8480
  Platform:
    osname=linux
    osvers=6.1.0-3-amd64
    archname=x86_64-linux
    uname='linux carme 6.1.0-3-amd64 #1 smp preempt_dynamic debian 6.1.8-1 (2023-01-29) x86_64 gnulinux '
    config_args='-des -Dusedevel'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2'
    cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='13.1.0'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/x86_64-linux-gnu /usr/lib /usr/lib64
    libs=-lpthread -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
    libc=/lib/x86_64-linux-gnu/libc.so.6
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.37'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'


---
@INC for perl 5.39.0:
    lib
    /usr/local/lib/perl5/site_perl/5.39.0/x86_64-linux
    /usr/local/lib/perl5/site_perl/5.39.0
    /usr/local/lib/perl5/5.39.0/x86_64-linux
    /usr/local/lib/perl5/5.39.0

---
Environment for perl 5.39.0:
    HOME=/home/ntyni
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/usr/lib/libeatmydata
    LOGDIR (unset)
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[jkeenan]

I modified @ntyni's tail.txt to the following program:

$ cat gh-21240-readline.pl 
use Test::More;
use File::Temp;
plan tests => 4;

my $tmpfile = File::Temp->new('test-XXXX', UNLINK => 1);
my $x;

open my $wr, '>>', $tmpfile;
open my $rd, '<', $tmpfile;

ok(! -s $tmpfile, "start with empty file");
$x = readline $rd;
is($x, undef, "nothing to read yet");

$wr->autoflush(1);
print $wr "foobar\n";

sleep 1;
ok(-s $tmpfile, "print flushed to file");

$x = readline $rd;
is($x, "foobar\n", "readline got appended content");

I then bisected with this invocation:

perl Porting/bisect.pl \
--start=v5.37.3 \
--end=v5.37.4 \
-- ./perl -Ilib $P5P_DIR/gh-21240-readline.pl

The result confirmed that the breakage occurred here:

commit 80c1f1e45e8ef8c27d170fae7ade41971fe20218
Author:     Tony Cook <[email protected]>
AuthorDate: Tue Aug 16 15:52:04 2022 +1000
Commit:     Tony Cook <[email protected]>
CommitDate: Wed Aug 31 10:51:09 2022 +1000

    only clear the stream error state in readline() for glob()

@tonycoz, can you take a look? Thanks.

[tonycoz]

It's a direct consequence of fixing #20060 / https://bugs.debian.org/1016369 since we no longer clear eof.

I'm inclined to say this is not a bug, but a fix.

[ntyni]

Right, I didn't think this through. Clearing eof with seek($rd, 0, 1) or $rd->clearerr() helps. Sorry for the noise.