gnuplot-py-users Mailing List for Gnuplot.py

jer...@um... wrote:
> I am writing a python script where I would like to send a number of graphs to
> different files. The number of graphs is undetermined beforehand. I was
> wondering if there was a way to use a variable in g('set output "filename"').
> So I could output to something like filename1, filename2, etc. where i would
> use a variable for the numbers.

Sure.  For example, you can use Python's usual string interpolation:

    g('set output "filename-%d.png"' % (file_number))

You might also want to check out the hardcopy() method, which
automatically does the "set term", "set output", etc.:

    g.hardcopy(
        filename=('filename-%d.png' % file_number),
        terminal='png',
        )

See the file "termdefs.py" to see what terminal types are already defined.

Michael

I am writing a python script where I would like to send a number of graphs to
different files. The number of graphs is undetermined beforehand. I was
wondering if there was a way to use a variable in g('set output "filename"').
So I could output to something like filename1, filename2, etc. where i would
use a variable for the numbers.

Noel O'Boyle wrote:
> Please find attached an example program with extensive commenting
> describing its behaviour. (I use WinXP on a 256MB old Dell laptop)
> 
> Noel
> 
> On Mon, 2005-11-07 at 21:29 +0100, Michael Haggerty wrote:
> 
>>Noel O'Boyle wrote:
>>
>>>In order to avoid the use of temporary files, I have tried to pass data
>>>to gnuplot using the inline option in WinXP. This works fine for small
>>>amounts of data but for larger amounts, the complete amount of data is
>>>not passed.
>>>
>>>For example, I tried to send data for 500 hundred (x,y) data points, but
>>>only 355 lines were sent to gnuplot. It was possible to see this at the
>>>interactive gnuplot prompt, and actually to plot the 355 data points by
>>>entering 'e' (for "end" - part of gnuplot's inline data passing
>>>mechanism, I suppose).
>>>
>>>Is this a known feature of Windows/gnuplot-py/gnuplot? I wasn't sure how
>>>to track down this bug - I can send you an example program that
>>>reproduces this problem if you want.
>>
>>I am not familiar with this problem.  It would help if you could send an
>>example program.  At the least, I could try it under Linux to see if the
>>same problem occurs there.

Your test script runs without any problems on my Linux computer.  It
certainly seems like some kind of buffer limitation problem, probably
specific to Windows.  Since I don't use windows, I can't help you (maybe
some other Windows/Gnuplot.py user can help.

Michael

Noel O'Boyle wrote:
> I have two questions relating to writing graphs to disk. The first is
> whether it is possible to do so without first plotting the image on the
> screen.

On my gnuplot setup, I can accomplish this by typing

    Gnuplot.GnuplotOpts.default_term = 'unknown'

before instantiating the Gnuplot.Gnuplot object.  I don't know if that
is a generic feature of Gnuplot, though.

> The second is related to the following problem with using popen, with
> which I think you are familiar:
> 
> *************************************************
> import Gnuplot,os
> 
> error = 0
> for i in range(50):
>     g = Gnuplot.Gnuplot()
>     d = Gnuplot.Func("sin(x)")
>     g.plot(d)
>     g.hardcopy("gnuplotpy.png",terminal="png")
>     try:
>         assert os.path.getsize("gnuplotpy.png")>0
>     except AssertionError:
>         error += 1
> print error
> *************************************************

Yes, I'm not surprised by this error (given the one-way communication
between Gnuplot.py and gnuplot) but strangely enough, I can't reproduce
it here (gnuplot 4.0p0, Debian Linux), even when plotting much larger
datasets.

> There seems to be no way around this using the popen family of commands
> apart from waiting until the file is created:
> *************************************************
>     size = os.path.getsize("gnuplotpy.png")
>     while size==0:
>         time.sleep(0.2)
>         size = os.path.getsize("gnuplotpy.png")
> *************************************************

Even this is not a solution, because the file could only be half-written
when your loop is done.

> If only Gnuplot returned an exit signal to the calling program, then you
> could use popen3 or something and wait for stdout to be closed.
> Unfortunately, this doesn't appear to be the case.

The only ideas that occur to me are:

1. Use popen3 and look for gnuplot's "gnuplot> " prompt before
continuing.  But I'm not sure that would work 100%.

2. Have gnuplot write its output to a named pipe with a Python listener.
 Gnuplot.py should only proceeds after the listener detects the file
close from the writer.  The listener would have to dump whatever it read
into the *real* file.  (Yucky and unportable.)

> On the other hand, the following approach addresses both of the problems
> listed above, and I am afraid that it is difficult for me to convince
> myself that using gnuplot-py would be better (for me):
> 
> *************************************************
> import os
> 
> errors = 0
> for i in range(50):
>     cmdfile = open("cmd.txt","w")
>     cmdfile.write('set terminal png\nset output "gnuplotpy.png"\n')
>     cmdfile.write("plot sin(x)\n")
>     cmdfile.close()
>     os.system("pgnuplot.exe cmd.txt")
>     try:
>         assert os.path.getsize("gnuplotpy.png")>0
>     except AssertionError:
>         errors += 1
> 
> print errors
> *************************************************
> 
> errors is always zero, as os.system does not return until the command is
> completed.
> 
> Would you consider including os.system in some way in gnuplot-py?

Are you sure you are really solving the problem, and not just hiding it
because your test script takes longer to run?  Given that you only saw
the problem 27/50 times with the original script (and I didn't see it at
all), it seems that the timing of this error is just barely in the
danger zone.

Second, could you solve your problem more easily in the original script
by inserting "g.close()" or "del g" after g.hardcopy()?  It might be
that the gnuplot process doesn't shut itself down until all commands are
complete.

Finally, if you want to do the cmd.txt approach, you can still construct
the command file (if you prefer) using

    g = Gnuplot.Gnuplot(filename='cmd.txt')

I only use Gnuplot.py myself very rarely now (my work doesn't call for
it) and I don't have time to enhance Gnuplot.py properly anymore, so I
won't be able to build other approaches into Gnuplot.py in the
foreseeable future.  Sorry.

Michael

Dear Michael,

I have two questions relating to writing graphs to disk. The first is
whether it is possible to do so without first plotting the image on the
screen.

The second is related to the following problem with using popen, with
which I think you are familiar:

*************************************************
import Gnuplot,os

error = 0
for i in range(50):
    g = Gnuplot.Gnuplot()
    d = Gnuplot.Func("sin(x)")
    g.plot(d)
    g.hardcopy("gnuplotpy.png",terminal="png")
    try:
        assert os.path.getsize("gnuplotpy.png")>0
    except AssertionError:
        error += 1
print error
*************************************************

On one run, 27 AssertionErrors were raised out of 50, the problem being
that the Python program continues while Gnuplot is still trying to plot
the image. This is a particular problem when the Python program ends
soon after calling Gnuplot, as the Gnuplot image is often not written to
disk.

There seems to be no way around this using the popen family of commands
apart from waiting until the file is created:
*************************************************
    size = os.path.getsize("gnuplotpy.png")
    while size==0:
        time.sleep(0.2)
        size = os.path.getsize("gnuplotpy.png")
*************************************************

If only Gnuplot returned an exit signal to the calling program, then you
could use popen3 or something and wait for stdout to be closed.
Unfortunately, this doesn't appear to be the case.

On the other hand, the following approach addresses both of the problems
listed above, and I am afraid that it is difficult for me to convince
myself that using gnuplot-py would be better (for me):

*************************************************
import os

errors = 0
for i in range(50):
    cmdfile = open("cmd.txt","w")
    cmdfile.write('set terminal png\nset output "gnuplotpy.png"\n')
    cmdfile.write("plot sin(x)\n")
    cmdfile.close()
    os.system("pgnuplot.exe cmd.txt")
    try:
        assert os.path.getsize("gnuplotpy.png")>0
    except AssertionError:
        errors += 1

print errors
*************************************************

errors is always zero, as os.system does not return until the command is
completed.

Would you consider including os.system in some way in gnuplot-py?

Regards,
Noel

On Mon, 2005-11-07 at 22:17 +0100, Michael Haggerty wrote:
> Noel,
> 
> Thanks again for the patch, and the additional information about allowed
> characters.
> 
> It seems to me that *all* characters can get through if we use
> double-quoted strings and simply backslash-quote '"' and '\'.  So I just
> added a function in the gp.py module called double_quote_string() that
> does just this, and use this method in the definitions of the two
> get_base_command_string() methods that return filenames.  These changes
> are committed into the main branch of CVS.
> 
> This seems to work under Linux.  Would you please check whether they
> work under Windows, and whether you can see any other problems with this
> solution?

I cannot see any problems with the fix and it works for me under
windows. A big thanks on behalf of all people with funny characters in
their name.

Regards,
Noel

> Noel O'Boyle wrote:
> > On Windows, if your user name contains an apostrophe, the use of
> > temporary files causes an error, as gnuplot uses apropostrophes (i.e.
> > single quotes) to delimit the filename.

Please find attached an example program with extensive commenting
describing its behaviour. (I use WinXP on a 256MB old Dell laptop)

Noel

On Mon, 2005-11-07 at 21:29 +0100, Michael Haggerty wrote:
> Noel O'Boyle wrote:
> > In order to avoid the use of temporary files, I have tried to pass data
> > to gnuplot using the inline option in WinXP. This works fine for small
> > amounts of data but for larger amounts, the complete amount of data is
> > not passed.
> > 
> > For example, I tried to send data for 500 hundred (x,y) data points, but
> > only 355 lines were sent to gnuplot. It was possible to see this at the
> > interactive gnuplot prompt, and actually to plot the 355 data points by
> > entering 'e' (for "end" - part of gnuplot's inline data passing
> > mechanism, I suppose).
> > 
> > Is this a known feature of Windows/gnuplot-py/gnuplot? I wasn't sure how
> > to track down this bug - I can send you an example program that
> > reproduces this problem if you want.
> 
> I am not familiar with this problem.  It would help if you could send an
> example program.  At the least, I could try it under Linux to see if the
> same problem occurs there.
> 
> Michael
> 
> 
> -------------------------------------------------------
> SF.Net email is sponsored by:
> Tame your development challenges with Apache's Geronimo App Server. Download
> it for free - -and be entered to win a 42" plasma tv or your very own
> Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
> _______________________________________________
> Gnuplot-py-users mailing list
> Gnu...@li...
> https://lists.sourceforge.net/lists/listinfo/gnuplot-py-users

I work on Mandrake linux with Mozilla Thunderbird and the last mail of
Michael is "ind=E9sirable" (in french) "a bad mail"
It is the first mail I receive with this logo
Why ???

>"Gnuplot" and the directory *containing* the Gnuplot directory has to be
>in the python path.  For example, if the python path includes /my/path,
>then your installation should put files in places like
>
>/my/path/Gnuplot/__init__.py
>/my/path/Gnuplot/_Gnuplot.py
>/my/path/Gnuplot/Data.py
>
>et cetera.
>
>It looks like you added the directory "/my/path/Gnuplot" to your python
>path, which is incorrect.  (You will also have to undo your tweaks.)
>
>I'm sending a CC of this list to the gnuplot mailing list.  Please use
>the mailing list for future questions.
>
>Michael
>
>
>-------------------------------------------------------
>SF.Net email is sponsored by:
>Tame your development challenges with Apache's Geronimo App Server. Down=
load
>it for free - -and be entered to win a 42" plasma tv or your very own
>Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
>_______________________________________________
>Gnuplot-py-users mailing list
>Gnu...@li...
>https://lists.sourceforge.net/lists/listinfo/gnuplot-py-users
>

John wrote:
> I have something of a problem with your otherwise lovely python
> interface ( v.1.7 on Mandriva Linux )
> 
> Using install with --home set to the directory I want the resultant
> files in, I get lots of problems with Gnuplot.py not being found. I had
> a quick look on the mailing list, and it was all as a result of iffy
> installs. The software appears to have installed correctly, and renaming
> _Gnuplot.py to Gnuplot.py alleviated many of the problems after a little
> tweaking of the funcutils.py file, but test.py still has trouble:
> 
> AttributeError: class Gnuplot has not attribute 'Data'
> 
> this from the line:
> 
> return apply(Gnuplot.Data, (xvals, data), keyw)
> 
> in funcutils.py
> 
> Any ideas?

Gnuplot.py is a module; it has to be located in a directory called
"Gnuplot" and the directory *containing* the Gnuplot directory has to be
in the python path.  For example, if the python path includes /my/path,
then your installation should put files in places like

/my/path/Gnuplot/__init__.py
/my/path/Gnuplot/_Gnuplot.py
/my/path/Gnuplot/Data.py

et cetera.

It looks like you added the directory "/my/path/Gnuplot" to your python
path, which is incorrect.  (You will also have to undo your tweaks.)

I'm sending a CC of this list to the gnuplot mailing list.  Please use
the mailing list for future questions.

Michael

Noel,

Thanks again for the patch, and the additional information about allowed
characters.

It seems to me that *all* characters can get through if we use
double-quoted strings and simply backslash-quote '"' and '\'.  So I just
added a function in the gp.py module called double_quote_string() that
does just this, and use this method in the definitions of the two
get_base_command_string() methods that return filenames.  These changes
are committed into the main branch of CVS.

This seems to work under Linux.  Would you please check whether they
work under Windows, and whether you can see any other problems with this
solution?

Thanks,
Michael

Noel O'Boyle wrote:
> On Windows, if your user name contains an apostrophe, the use of
> temporary files causes an error, as gnuplot uses apropostrophes (i.e.
> single quotes) to delimit the filename.

Noel O'Boyle wrote:
> In order to avoid the use of temporary files, I have tried to pass data
> to gnuplot using the inline option in WinXP. This works fine for small
> amounts of data but for larger amounts, the complete amount of data is
> not passed.
> 
> For example, I tried to send data for 500 hundred (x,y) data points, but
> only 355 lines were sent to gnuplot. It was possible to see this at the
> interactive gnuplot prompt, and actually to plot the 355 data points by
> entering 'e' (for "end" - part of gnuplot's inline data passing
> mechanism, I suppose).
> 
> Is this a known feature of Windows/gnuplot-py/gnuplot? I wasn't sure how
> to track down this bug - I can send you an example program that
> reproduces this problem if you want.

I am not familiar with this problem.  It would help if you could send an
example program.  At the least, I could try it under Linux to see if the
same problem occurs there.

Michael

Dear all,

In order to avoid the use of temporary files, I have tried to pass data
to gnuplot using the inline option in WinXP. This works fine for small
amounts of data but for larger amounts, the complete amount of data is
not passed.

For example, I tried to send data for 500 hundred (x,y) data points, but
only 355 lines were sent to gnuplot. It was possible to see this at the
interactive gnuplot prompt, and actually to plot the 355 data points by
entering 'e' (for "end" - part of gnuplot's inline data passing
mechanism, I suppose).

Is this a known feature of Windows/gnuplot-py/gnuplot? I wasn't sure how
to track down this bug - I can send you an example program that
reproduces this problem if you want.

Regards,
Noel

Juho Schultz wrote:
> Or you could use raw strings:
> 
> r"c:\docume~1\noelo'~1\locals~1\temp\tmpcxnwsm"
> 
> So python treats \ in the string just like any other character.

The problem isn't Python's string translation, it is gnuplot's.  The
situation is that there is a valid python string containing a filename,
and that filename has to be passed to gnuplot quoted in such a way that
gnuplot interprets it as the original string.

For all characters except "'" (single-quote), simply single-quoting the
string to gnuplot works.  But there doesn't seem to be any way to pass a
single quote to gnuplot within a single-quoted string (e.g., '\''
results in a two-character string containing "\" and "'").

So if we need to support filenames with single-quotes, we have to use
gnuplot's double-quoted format.

I think double-quoted strings should work OK, but we have to quote both
'"' (double-quote) and "\" (backslash).  It might not be necessary to
handle double-quote for windows filenames, but we should handle it
anyway for generality.  Then, I believe, we could use the same quoting
code for all operating systems.

Michael

On Mon, 7 Nov 2005, Noel O'Boyle wrote:

>Problem:
>
>On Windows, if your user name contains an apostrophe, the use of
>temporary files causes an error, as gnuplot uses apropostrophes (i.e.
>single quotes) to delimit the filename.
>
>gnuplot> plot 'c:\docume~1\noelo'~1\locals~1\temp\tmpcxnwsm' notitle
>                                   ^
>         invalid character
>
>
>Solution:
>
>If you replace the single quotes with double quotes, gnuplot is happy
>with the apostrophe but misinterprets the backslashes as escape
>character sequences.
>
>gnuplot> plot "c:\docume~1\noelo'~1\locals~1\temp\tmpcxnwsm" notitle
>              ^
>         can't read data file "c:docume~1
>oelo'~1locals~1 emp     mpcxnwsm"
>
>....so you need to also replace all of the backslashes with forward
>slashes.

Or you could use raw strings:

r"c:\docume~1\noelo'~1\locals~1\temp\tmpcxnwsm"

So python treats \ in the string just like any other character.


-- 

   Juho Schultz

   e-mail: juh...@as...
   www.astro.helsinki.fi/~jschultz
   Observatory
   P.O. Box 14
   FIN-00014 University of Helsinki
   FINLAND

On Mon, 2005-11-07 at 12:14 +0100, Michael Haggerty wrote:

> Hello,
> 
> Thanks for the information and the patch.
> 
> Won't your patch create new problems for filenames that contain
> double-quotes?  Or is that somehow forbidden?

> What if a filename contains single-quotes *and* double-quotes?
> 
> In other words, is there a way to solve this problem in such a way
that
> it works for *all* legitimate Windows filenames?
I have been using this method for my own program's link to Gnuplot (in
gausssum.sf.net) for the last year or so, and there have been no
reported problems (either in Windows or Linux or Macs). Which may just
mean that I have no users, of course :-)

The allowed characters are listed here (if you trust Wikipedia):
http://en.wikipedia.org/wiki/Filename

All listed versions of Windows as well as DOS, disallow " in filenames.
Win95 and WinXP allow ' in filenames. Mac and Unix allow any characters
except : and / respectively.


> Regarding your specific proposed patch:
> 
> 1. I think the conversion of filenames should happen via a call to the
> os-specific code.
Possibly, but I think the problem is gnuplot specific, not OS specific.
Although, I need to test this.

I'm in Linux at the moment but I'm not sure how to set up a proper test.
I have created files in folders called 'test and "test, but I'm not sure
how to go about testing the patch, unless there's an easy way to
redirect mkstemp. Maybe changing the TMP environment variable?

> 2. string.replace() would be easier than an explicit loop.
That's true.

Noel

Noel O'Boyle wrote:
> Problem:
> 
> On Windows, if your user name contains an apostrophe, the use of
> temporary files causes an error, as gnuplot uses apropostrophes (i.e.
> single quotes) to delimit the filename.
> 
> gnuplot> plot 'c:\docume~1\noelo'~1\locals~1\temp\tmpcxnwsm' notitle
>                                    ^
>          invalid character
> 
> 
> Solution:
> 
> If you replace the single quotes with double quotes, gnuplot is happy
> with the apostrophe but misinterprets the backslashes as escape
> character sequences.
> 
> gnuplot> plot "c:\docume~1\noelo'~1\locals~1\temp\tmpcxnwsm" notitle
>               ^
>          can't read data file "c:docume~1
> oelo'~1locals~1 emp     mpcxnwsm"
> 
> ....so you need to also replace all of the backslashes with forward
> slashes.
> 
> gnuplot> plot "c:/docume~1/noelo'~1/locals~1/temp/tmpcxnwsm"
> notitle            
> 
> **Success!!**

Hello,

Thanks for the information and the patch.

Won't your patch create new problems for filenames that contain
double-quotes?  Or is that somehow forbidden?

What if a filename contains single-quotes *and* double-quotes?

In other words, is there a way to solve this problem in such a way that
it works for *all* legitimate Windows filenames?

Regarding your specific proposed patch:

1. I think the conversion of filenames should happen via a call to the
os-specific code.

2. string.replace() would be easier than an explicit loop.

Michael

Noel O'Boyle wrote:
> It seems that at the moment, it is only possible to save images as .ps.
> This is a problem for any Python programs that want to display the image
> (e.g. using Tkinter).
> 
> Would it be difficult to add .png support?

It is already possible to use

    g.hardcopy(filename='myfile.png', terminal='png')

with some other option possibilities (fontsize, monochrome, etc) defined
in termdefs.py within terminal_opts['png'].

Is this what you need?

Michael

Either go back to version 1.6 of browse in this mailing list, there is a
line that you can change in the configuration that solve this issue on
some occasions. The problem of mouse interactions of gnuplot4 through
gnuplot-py raised more then once.

   Nadav.

Will Muir wrote:

>How can I make the mouse zoom feature work through the python interface
>as it does when a just run gnuplot stand alone.
>
>Will
>
>
>
>-------------------------------------------------------
>SF.Net email is sponsored by:
>Tame your development challenges with Apache's Geronimo App Server. Download
>it for free - -and be entered to win a 42" plasma tv or your very own
>Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
>_______________________________________________
>Gnuplot-py-users mailing list
>Gnu...@li...
>https://lists.sourceforge.net/lists/listinfo/gnuplot-py-users
>
>  
>

It seems that at the moment, it is only possible to save images as .ps.
This is a problem for any Python programs that want to display the image
(e.g. using Tkinter).

Would it be difficult to add .png support?

Regards,
Noel.

Dear all,

Please find attached a patch for the following problem:

Problem:

On Windows, if your user name contains an apostrophe, the use of
temporary files causes an error, as gnuplot uses apropostrophes (i.e.
single quotes) to delimit the filename.

gnuplot> plot 'c:\docume~1\noelo'~1\locals~1\temp\tmpcxnwsm' notitle
                                   ^
         invalid character


Solution:

If you replace the single quotes with double quotes, gnuplot is happy
with the apostrophe but misinterprets the backslashes as escape
character sequences.

gnuplot> plot "c:\docume~1\noelo'~1\locals~1\temp\tmpcxnwsm" notitle
              ^
         can't read data file "c:docume~1
oelo'~1locals~1 emp     mpcxnwsm"

....so you need to also replace all of the backslashes with forward
slashes.

gnuplot> plot "c:/docume~1/noelo'~1/locals~1/temp/tmpcxnwsm"
notitle            

**Success!!**

Patch Attached:
mishandledfilenames.txt

How can I make the mouse zoom feature work through the python interface
as it does when a just run gnuplot stand alone.

Will