expectperl-discuss Mailing List for Expect.pm

On Fri, May 28, 2004 at 11:35:24AM -0500, Peter Eisch wrote:
> 
> Yes, the difference is whether it runs as root or not.  In fact, if I pull
> out all the tty calls except for raw_tty(1) and don't run as root it all
> chugs along just fine.
> 
> My calls to expect() were messy to, my lesson for 'self' this morning is
> don't expect '' before sending the command, just stinking send the command.
> 
> It just started working here in the last 20 mins and I have some stuff to do
> before I can get back to cleaning this up, but I'll fire off a redux when I
> can.
> 

	Btw, what type of system are you trying to run this on?

	Austin

Yes, the difference is whether it runs as root or not.  In fact, if I pull
out all the tty calls except for raw_tty(1) and don't run as root it all
chugs along just fine.

My calls to expect() were messy to, my lesson for 'self' this morning is
don't expect '' before sending the command, just stinking send the command.

It just started working here in the last 20 mins and I have some stuff to do
before I can get back to cleaning this up, but I'll fire off a redux when I
can.

Thanks all,

peter

> From: Roland Giersig <RGi...@cp...>
> Date: Fri, 28 May 2004 10:45:02 +0200
> To: exp...@li...
> Subject: Re: [Expectperl-discuss] My tty drama
> 
>> Note that if I ran it from the command-line (real tty) it ran fine.
> 
> Yes, thats one thing that is still baffling me. You see, I modelled the
> pty allocation code mostly after sshd, so it should work to capture even
> /dev/tty of the spawned process, but it sometimes (on some systems, as a
> daemon, without a user tty) doesn't. Unfortunately, sshd runs as root,
> so it can do some things that normal processes cannot do, and I left out
> those things (if I remember correctly, it has been a while). And pty
> allocation *is* black magic, very heavy black magic indeed.
> 
> So, the short answer is: sorry, I don't know why it doesn't work. And
> that's the reason why I always ask people if there is a way to solve the
> problem without using Expect, using ssh/scp/ncftp/Net::Telnet etc.
> instead...
> 
> Roland
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g.
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
> _______________________________________________
> Expectperl-discuss mailing list
> Exp...@li...
> https://lists.sourceforge.net/lists/listinfo/expectperl-discuss
> 

> Note that if I ran it from the command-line (real tty) it ran fine.

Yes, thats one thing that is still baffling me. You see, I modelled the 
pty allocation code mostly after sshd, so it should work to capture even 
/dev/tty of the spawned process, but it sometimes (on some systems, as a 
daemon, without a user tty) doesn't. Unfortunately, sshd runs as root, 
so it can do some things that normal processes cannot do, and I left out 
those things (if I remember correctly, it has been a while). And pty 
allocation *is* black magic, very heavy black magic indeed.

So, the short answer is: sorry, I don't know why it doesn't work. And 
that's the reason why I always ask people if there is a way to solve the 
problem without using Expect, using ssh/scp/ncftp/Net::Telnet etc. 
instead...

Roland

>> [ Hey, for those that only know sermons and not gospel,  point me to the
>> FTPS.pm and I'll gladly use that then suffer through this drama. ]
> 
> Maybe you can make one using Expect so the rest of us can avoid
> the suffering.
> 

Certainly.

...
>> use Carp;
>> use Expect;
> 
> Toss a little debugging in.
> $Expect::Debug=1;
> $Expect::Exp_Internal=1;
>

Cool...

> You're creating a bit of extra work here by not creating a lexical to
> hold $self->{_exp}. It looks like this fouls you up below.
> Try this:
> 
>>     $self->{_exp} = new Expect;
> $expect = $self->{_exp};
> 

Seems reasonable to me.

...
>> 
>>     $self->{_exp}->expect($self->{_timeout},
>>                  [ qr/Name /i, sub { my $self = shift;
>>                                           $self->send($user . "\n");
>                                           ^^^^^^^^^^^
> 
> Does '$self' have a send method? $self->{_exp}->send()? or, better
> yet, $expect->send().

The 'my $self = shift; $self->send()' all happens within the instance of
Expect.  The little trick of using $self within a sequence seems pretty
nifty, but I've changed it to $exp...  the result is the same.

> 
>>                                           exp_continue; }],
>>                  [ qr/Password:/i, sub { my $self = shift;
>>                                           $self->send($pass . "\n");
>                                           ^^^^^^^^^^^
>>                                           exp_continue; }],
>>                  $self->{_prompt});
> 
> 
> Try fixing that, and turn on debugging.
> 

Note that if I ran it from the command-line (real tty) it ran fine.


For example, when I run it as a daemon the log file found in /tmp/exp-$$
only logs this:

[root@fubar root]# cat /tmp/exp-25225
Connected to secureftp.some.gvmt.office.
220- Department of Human Service Secure Transport site!
220-
220 Secure FTP Server ready.


If I run it from the command line:

[root@fubar root]# cat /tmp/exp-18484
Connected to secureftp.some.gvmt.office.
220- Department of Human Service Secure Transport site!
220-
220 Secure FTP Server ready.
Name (secureftp.some.gvmt.office:root): 234 SSLv23/TLSv1
[TLSv1/SSLv3, cipher DES-CBC3-SHA, 168 bits]
331 Password required for username.
Password:
230 Virtual user username logged in.
235 PBSZ=0
200 PROT command successful
TLS/SSL protection of data connections on.
Remote system type is UNIX.
Using binary mode to transfer files.
ftps> 250 CWD command successful.
ftps> 200 PORT command successful.
150 Opening ASCII mode SSL data connection for file list.
total 353677
-rwxrwxrwx   1 root     2000     61009251 May 20 08:54
965713400_RiskAdj_20040519.dat
-rwxrwxrwx   1 root     2000     59063559 Feb 20 15:42
965713400_RiskAdj_risk.dat
-rwxrwxrwx   1 root     2000     61009251 May 18 17:24
965713400_RiskAdj_risk_051904.dat
226 Transfer complete.
ftps> local: 
/home/rs/tp/username/to-username/.965713400_RiskAdj_20040519.dat remote:
965713400_RiskAdj_20040519.dat
200 PORT command successful.
150 Opening BINARY mode SSL data connection for
965713400_RiskAdj_20040519.dat.
  1%    648 KB    06:03 ETA



Yeah, I can turn off the verbose stuff at some point...

Other thoughts?

Thanks,

peter

On Thu, May 27, 2004 at 11:11:21AM -0500, Peter Eisch wrote:
> 
> I'm having the age-old problem tty issues where when I run the script from
> the command-line things work like they should.  If it runs from the daemon
> it doesn't work right though the script itself seems to run.  The client is
> bsdftps, er ftps from the command-line.
> 
> [ Hey, for those that only know sermons and not gospel,  point me to the
> FTPS.pm and I'll gladly use that then suffer through this drama. ]

	Maybe you can make one using Expect so the rest of us can avoid
the suffering.

> 
> My relevant portions:
> 
> use POSIX ":sys_wait_h";
> use FileHandle;
> use IO::File;
> use IO::Select;
> use IO::Pty;
> use Carp;
> use Expect;

 Toss a little debugging in.
$Expect::Debug=1;
$Expect::Exp_Internal=1;

> 
>     $Expect::Log_Stdout=0;

You're creating a bit of extra work here by not creating a lexical to
hold $self->{_exp}. It looks like this fouls you up below.
Try this:

>     $self->{_exp} = new Expect;
$expect = $self->{_exp};

>     $self->{_exp}->raw_pty(1);
>     $self->{_exp}->stty('raw');
>     $self->{_prompt} = 'ftps>';
>     $self->{_timeout} = 10;
>     my @cmd = '/usr/bin/ftps';
>     push @cmd, '-e';
>     push @cmd, '-v';
>     push @cmd, '-P', $self->{_port};
>     push @cmd, '-d' if ($self->{_debug});
>     push @cmd, '-p' if ($self->{_passive});
>     push @cmd, '-n' if ($self->{_autologin});
>     push @cmd, $self->{'_host'};
>     my $c = join ' ', @cmd;
>     VSI::RS::Results->new($ident, 0, '', '', 0)->logThis('debug', "STARTUP:
> $c") if ($DEBUG);
> 
>     close(STDOUT);
>     open(STDOUT, '>/dev/null');
>     close(STDIN);
>     close(STDERR);
>     $self->{_exp}->log_file("/tmp/exp-$$", 'w');
> 
>     #$self->{_exp}->slave->set_raw();
>     #$self->{_exp}->set_raw();
> 
>     $self->{_exp}->spawn(@cmd) ||
>         return 0;
> 
>     $self->{_exp}->expect($self->{_timeout},
>                  [ qr/Name /i, sub { my $self = shift;
>                                           $self->send($user . "\n");
                                            ^^^^^^^^^^^

	Does '$self' have a send method? $self->{_exp}->send()? or, better
yet, $expect->send().

>                                           exp_continue; }],
>                  [ qr/Password:/i, sub { my $self = shift;
>                                           $self->send($pass . "\n");
                                            ^^^^^^^^^^^
>                                           exp_continue; }],
>                  $self->{_prompt});


	Try fixing that, and turn on debugging.

	Good luck,

	Austin

I'm having the age-old problem tty issues where when I run the script from
the command-line things work like they should.  If it runs from the daemon
it doesn't work right though the script itself seems to run.  The client is
bsdftps, er ftps from the command-line.

[ Hey, for those that only know sermons and not gospel,  point me to the
FTPS.pm and I'll gladly use that then suffer through this drama. ]

My relevant portions:

use POSIX ":sys_wait_h";
use FileHandle;
use IO::File;
use IO::Select;
use IO::Pty;
use Carp;
use Expect;

    $Expect::Log_Stdout=0;
    $self->{_exp} = new Expect;
    $self->{_exp}->raw_pty(1);
    $self->{_exp}->stty('raw');
    $self->{_prompt} = 'ftps>';
    $self->{_timeout} = 10;
    my @cmd = '/usr/bin/ftps';
    push @cmd, '-e';
    push @cmd, '-v';
    push @cmd, '-P', $self->{_port};
    push @cmd, '-d' if ($self->{_debug});
    push @cmd, '-p' if ($self->{_passive});
    push @cmd, '-n' if ($self->{_autologin});
    push @cmd, $self->{'_host'};
    my $c = join ' ', @cmd;
    VSI::RS::Results->new($ident, 0, '', '', 0)->logThis('debug', "STARTUP:
$c") if ($DEBUG);

    close(STDOUT);
    open(STDOUT, '>/dev/null');
    close(STDIN);
    close(STDERR);
    $self->{_exp}->log_file("/tmp/exp-$$", 'w');

    #$self->{_exp}->slave->set_raw();
    #$self->{_exp}->set_raw();

    $self->{_exp}->spawn(@cmd) ||
        return 0;

    $self->{_exp}->expect($self->{_timeout},
                 [ qr/Name /i, sub { my $self = shift;
                                          $self->send($user . "\n");
                                          exp_continue; }],
                 [ qr/Password:/i, sub { my $self = shift;
                                          $self->send($pass . "\n");
                                          exp_continue; }],
                 $self->{_prompt});

    my $return = $self->{_exp}->before();
    foreach my $line (split /\r?\n/, $return) {
        $self->message($line)
             if ($line =~ /^(\d+) /);
    }

All of the above works to where it can connect and log in.  So then I set to
the directory with:

    my $cmd = "cd $dir\n";

    $self->{_exp}->expect($self->{_timeout},
                '', $self->{_exp}->send($cmd), $self->{_prompt});

    my $return = $self->{_exp}->before();

    foreach my $line (split /\r?\n/, $return) {
        $self->message($line)
             if ($line =~ /^(\d+) /);
    }

But I never get back the junk I should after the command is issued.  Oddly,
my prompt matches.  Assuming though that the 'cd' worked, I'm getting
nothing back when I do a listing in the directory:

    my $cmd = "dir\n";
    my @list;
    my @listing;
    
    $self->{_exp}->expect($self->{_timeout},
                '', $self->{_exp}->send($cmd), $self->{_prompt});

    my $return = $self->{_exp}->before();
    foreach my $line (split /\r?\n/, $return) {
    
          my @fields = split /\s+/, $line;
          if ((scalar @fields) >= 9 && $line!~/^\d{3}/) {
              my $index = (scalar @list);
              $list[$index]{'perm'} = shift @fields; # perms
              shift @fields; # skip weird thing
              $list[$index]{'user'} = shift @fields;
              $list[$index]{'group'} = shift @fields;
              $list[$index]{'size'} = shift @fields;
              $list[$index]{'date'} = join ' ', (shift @fields, shift
@fields, shift @fields);
              $list[$index]{'filename'} = join ' ', @fields;
#               push @listing, $list[$index]{'filename'};
              $listing[$index] = $list[$index]{'filename'};
          } else {
              $self->message($line)
                  if ($line =~ /(\d+) /);
          }
    } 
    return @listing;

Again, if I run the same process without being invoked underneath (RH7.3):

  daemon --user genuser ...

It all works swell.

What [simple] detail am I missing?

Many thanks,

peter

On Wed, May 26, 2004 at 09:13:27AM -0400, John Wingenbach wrote:
> 
> Note that the creation of the slave (assuming the if test fails), is 
> dependent upon retrieving the ttyname.  However, Expect.pm "use"s both 
> POSIX and IO::Pty.  The problem with the default 'use POSIX;' is that 
> POSIX also has a ttyname declaration and thus once an Expect object is 
> blessed, the ttyname function used is NOT the IO::Pty one.  The solution 
> is to limit what is being incorporated into Expect from the POSIX 
> module.  From my limited tests, I have been able successfully change the 
> use line to:
> 

	Seems to me, after thinking about this for a bit, that
IO::Pty::ttyname and POSIX::ttyname should do exactly the same thing. If
POSIX::ttyname isn't returning correctly it seems like we should patch it.
	One of the issues is that it probably doesn't like filehandle refs,
but that would probably be an easy fix.

	Austin

On Wed, May 26, 2004 at 03:40:50PM +0200, Roland Giersig wrote:
> Thanks for the fix. I hope I will find the time to check what exactly is 
> needed from the POSIX module (can't remember offhand).
> 

	setsid(), specifically. IO::Stty also uses POSIX for the termio
stuff.

	Austin

Thanks for the fix. I hope I will find the time to check what exactly is 
needed from the POSIX module (can't remember offhand).

Unfortunately my current working and personal situation does not leave 
time for much maintainance, but rest assured that Expect and IO::Pty are 
NOT orphaned. :o)

I will try to create a new release in the next few days (there are 
several fixes waiting to be released).

Roland


John Wingenbach wrote:

> After much fun, I have found the solution to a bug in Expect.pm.  The 
> problem surfaces in the attempt to retrieve a slave (e.g. 
> clone_winsize_from):
> 
> 
> from IO::Pty:
> 
> sub slave {
>  @_ == 1 or croak 'usage: $pty->slave();';
> 
>  my $master = shift;
> 
>  if (exists ${*$master}{'io_pty_slave'}) {
>    return ${*$master}{'io_pty_slave'};
>  }
> 
>  my $tty = $master->ttyname();
> 
>  my $slave = new IO::Tty;
> 
>  $slave->open($tty, O_RDWR | O_NOCTTY) ||
>    croak "Cannot open slave $tty: $!";
> 
>  return $slave;
> }
> 
> 
> Note that the creation of the slave (assuming the if test fails), is 
> dependent upon retrieving the ttyname.  However, Expect.pm "use"s both 
> POSIX and IO::Pty.  The problem with the default 'use POSIX;' is that 
> POSIX also has a ttyname declaration and thus once an Expect object is 
> blessed, the ttyname function used is NOT the IO::Pty one.  The solution 
> is to limit what is being incorporated into Expect from the POSIX 
> module.  From my limited tests, I have been able successfully change the 
> use line to:
> 
> use POSIX qw (WNOHANG);
> 
> With this change, the ttyname function can be called and the name 
> retrieved.  Thus, window resizing:
> 
> $e = new Expect;
> $e->slave->clone_winsize_from(\*STDIN);
> $SIG{WINCH} = \&winch;
> 
> 
> sub winch {
>  $e->slave->clone_winsize_from(\*STDIN);
>  kill WINCH => $e->pid if $e->pid;
>  $SIG{WINCH} = \&winch;
> }
> 
> 
> works as well as any other call which needs to retrieve the ttyname from 
> the expect object.
> 
> Cheers,
> 
> John C. Wingenbach
> 
> 
> PS... I am assuming that the authors/maintainers of Expect are no longer 
> active as I have not seen any kind of involvement from them on the lists 
> nor on CPAN.
> 
> 
> Is Expect.pm a dead or orphaned module?
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g. 
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
> _______________________________________________
> Expectperl-discuss mailing list
> Exp...@li...
> https://lists.sourceforge.net/lists/listinfo/expectperl-discuss
> 

After much fun, I have found the solution to a bug in Expect.pm.  The 
problem surfaces in the attempt to retrieve a slave (e.g. 
clone_winsize_from):


from IO::Pty:

sub slave {
  @_ == 1 or croak 'usage: $pty->slave();';

  my $master = shift;

  if (exists ${*$master}{'io_pty_slave'}) {
    return ${*$master}{'io_pty_slave'};
  }

  my $tty = $master->ttyname();

  my $slave = new IO::Tty;

  $slave->open($tty, O_RDWR | O_NOCTTY) ||
    croak "Cannot open slave $tty: $!";

  return $slave;
}


Note that the creation of the slave (assuming the if test fails), is 
dependent upon retrieving the ttyname.  However, Expect.pm "use"s both 
POSIX and IO::Pty.  The problem with the default 'use POSIX;' is that 
POSIX also has a ttyname declaration and thus once an Expect object is 
blessed, the ttyname function used is NOT the IO::Pty one.  The solution 
is to limit what is being incorporated into Expect from the POSIX 
module.  From my limited tests, I have been able successfully change the 
use line to:

use POSIX qw (WNOHANG);

With this change, the ttyname function can be called and the name 
retrieved.  Thus, window resizing:

$e = new Expect;
$e->slave->clone_winsize_from(\*STDIN);
$SIG{WINCH} = \&winch;
 
 
sub winch {
  $e->slave->clone_winsize_from(\*STDIN);
  kill WINCH => $e->pid if $e->pid;
  $SIG{WINCH} = \&winch;
}


works as well as any other call which needs to retrieve the ttyname from 
the expect object.

Cheers,

John C. Wingenbach


PS... I am assuming that the authors/maintainers of Expect are no longer 
active as I have not seen any kind of involvement from them on the lists 
nor on CPAN.


Is Expect.pm a dead or orphaned module?

I'm currently writing a script that executes commands on a remote
system, I noticed the log files and the output file I generate contain
ctrl M chars. How do I disable this? $obj->raw_pty(0), doesn't seem to
work for me. Thanks!