New plotting style 'hsteps' and 'vsteps'

I have looked at the code itself only at a quick glance. The questions below
are from reading the About document and playing a little bit with the examples.

• Why is it necessary or useful to have a separate "width" keyword
  when you can simply place the same value in the "using" specifier?
  plot $data using 1:2:(width) with hsteps

• What is the intent or rationale for what happens when
  the width of a step, either from column 3 or from the "width" keyword,
  is greater than the interval to the adjacent point?

• I find the "y=NaN" option very confusing because it doesn't seem to
  have anything to do with y values. If I understand correctly, it
  really means "connect the endpoints of each step to the endpoints
  of the adjacent steps". Right? In which case I think it would be
  clearer to make it a separate keyword, perhaps a three-way choice
  {split|link|nolink}
  I don't love "split" as a keyword either, but I don't have a better
  suggestion at the moment. "boxes"? "drop"? "verticals"?

• I was confused by the effect of the fillstyle keyword "border",
  which seems not to apply separately to each fill box.
  I suppose this makes it more like "with filledcurves" than like
  "with boxes" or "with histograms", but it was not what I expected.

• How many of the existing plot styles steps/histeps/fsteps/fillsteps
  can be reproduced using the new hsteps style? Could we simplify the
  code base by emulating them? I don't mean to deprecate the older
  commands, just to implement them by replacing the existing code
  with a wrapper around the new code. Rather than having separate
  parameters hsteps_* in struct curve_points, define a new structure
  that holds these and any other flags that might be needed to emulate
  the old styles. Then the "plot" command can fill them in, place
  a pointer in the plot header, and funnel both the old and new styles
  through the new code.

• Do you see a way to combine the new option with histogram plotting
  so that it would satisfy previous feature request 425?
  https://sourceforge.net/p/gnuplot/feature-requests/425/

10 Aug 2023
Here is a second round of comments.

• Perhaps I am overlooking a relevant application, but I do not see a use
  for "split" mode. Why would one ever use this rather than "with boxes"?
  As I see it, "with hsteps" offers three useful modes so far.
  1) The current default that drops a vertical line either to the next
  step or to the baseline depending on step width.
  Perhaps if adjacent steps overlap (one is wider than expected)
  the vertical should stop when it hits the other step rather than
  continuing through it to hit the baseline.
  2) The current "nolink"
  3) The current "y=NaN" which I still think should instead be
  a separate keyword.
  So I suggest that these should be three mutually exclusive options
  plot DATA with hsteps {default|nolink|link}
  There might be better names for the modes, but I don't have any to offer
  at the moment.

• I convey a comment from geneticist colleague who uses gnuplot to
  prepare figures. She is particularly enthusiastic about the new
  capability shown in Figure 11 of your About document.

• The attached plot (png image) shows two behaviours that I find unexpected.
  Both are illustrated by the plot
  array A = [2, 1, 6, 3, 7, 4, 5, 3, 5, 2, 0, 5]
  plot A using 1:2:(1) with hsteps above y=2.5 fs solid 0.5 border

Autoscaling of the endpoints does not work as I would expect.
In particular xmax does not extend far enough to hold the full curve.
I think this can be fixed in plot2d:get_data() by correctly calculating
xlow and xhigh before passing them to store2d_point().

I do not think that above/below should affect the curve itself,
only the fill area.

Code

I have two overall comments on the new C code. Both are from the perspective of consistency with the existing gnuplot code base.

The first is that I would prefer to drop the idea of a separate "vsteps" plot style. Yes there have been requests dating back at least 20 years that gnuplot should provide some "plot everything sideways" option so that the independent variable runs vertically and the dependent variable runs horizontally. And maybe it would have been better to design it that way from the start. But duplicating large blocks of code for one plot style rather than addressing the program as a whole is IMHO not a good way to go. I prefer to admit that the original design was imperfect and accept those limitations. Gnuplot was not designed to produce vertical plots, just as gnuplot was not designed to provide true 3D graphics.

The second is that everywhere else in the gnuplot code, variables named point or points refer to a structure of type coordinate. E.g.
struct coordinate *points = plot->points;

However the new code for plot_hsteps() defines a different structure hstep_point that serves almost the same purpose but does not overlay struct coordinate. It then tries to hide the incompatibility by using #define macros to substitute map_u(points[i].ul) for what would normally be map_x(points[i].xlow) and so on. I find this very misleading when I try to read the code because the "points" structures are not compatible. Furthermore I worry that the code might break depending on the platform+configuration sizes of fields with type "coordval", "intgr", etc.

Please use the existing point coordinate structure and field names. If additional flag bits are needed (hstep_point.state) then for now you can use coordinate.extra. That field is new in version 6. The original idea was that it would be present or absent depending on the platform and used optionally if it was present. But it seems that new version 6 code is increasingly relying on this field to be present. Maybe it should be given a better name ("options"? "private"? "temp"?) and promoted to a permanent member of the structure that is always present.

A patch (v2) incorporating the content of the exchange so far and the revised documentation is attached.

The main changes in patch v1->v2 are as follows,

Specification

• remove vsteps style
• remove width keyword
• remove split keyword
• replace the role of y=NaN option to link keyword
• nolink also makes the baseline undefined

Code

• revert the coordinate designation (u,v) to (x,y)
• use coordinate struct instead of histep_point struct
• introduce of pilot implementation of steps/fsteps/fillsteps/histeps emulation

The following is a short reply to the comments so far.

The first is that I would prefer to drop the idea of a separate "vsteps" plot style.

I am in the field of meteorology, where I have the opportunity to chart the vertical profiles of weather elements. That's where I wanted to make use of vsteps. However, I would like to concentrate here on the introduction of hsteps and leave the proposal for vsteps for another time.

Please use the existing point coordinate structure and field names.

The coordinate struct is now used. Variables required at each point, such as 'state' and 'gap', are now allocated separately.

Why is it necessary or useful to have a separate "width" keyword when you can simply place the same value in the "using" specifier?

In the new patch, the width option has been removed.

Why would one ever use this rather than "with boxes"?

It was implemented with the intention of being used with vsteps.
But, in the new patch, the split option has been removed
along with the removal of vsteps.

So I suggest that these should be three mutually exclusive options
plot DATA with hsteps {default|nolink|link}

In the new patch, I use the name link instead of 'y=NaN'.

The 'link' option (originally y=NaN) makes the baseline undefined. In conjunction with this, the 'nolink' option has changed also to make the baseline undefined.

I do not think that above/below should affect the curve itself, only the fill area.

It has been fixed in the new patch.

Autoscaling of the endpoints does not work as I would expect. In particular xmax does not extend far enough to hold the full curve. I think this can be fixed in plot2d:get_data() by correctly calculating xlow and xhigh before passing them to store2d_point().

It has been partialy implemented in the new patch. If the width is given by 3rd column, xlow/xhigh is set in get_data() before passing them to store2d_point(). If the width is not explicitly given, autoscaling is not work correctly because of the lack of ability of the information of the adjacent segments in get_data().

How many of the existing plot styles steps/histeps/fsteps/fillsteps can be reproduced using the new hsteps style?

Pilot implementation of steps/fsteps/fillsteps/histeps emulation is included in the new patch. The original function plot_steps ... are left intact, so it is possible to immediately revert to the original behaviour.

I changed the above/below behavior as Ethan mentioned.
But, I want to revert it as in original patch (v1).
That is, the above/below option restricts the border drawing area as well as the fill area.
This is because expression through v1 behaviour is more flexible than v2 behaviour.

First of all, the second figure of Ethan's attached file 'above_below.png' is somewhat incorrect regarding the handling of baselines. The option 'above y=2.5' defines the lowest limit of filling at y=2.5 and simultainously it defines the baseline at y=2.5.

Anyway, the second figure in 'above_below.png' can be realised even in original patch (v1).

array A = [2, 1, 6, 3, 7, 4, 5, 3, 5, 2, 0, 5]
plot A using 1:2:(1) with hsteps above y=2.5 fs solid 0.5 noborder,\
     A using 1:2:(1) with hsteps lw 2

And original patch (v1) has the expression like the following,

set xrange [0:13]
array A = [2, 1, 6, 3, 7, 4, 5, 3, 5, 2, 0, 5]
plot A using 1:2:(1) with hsteps above y=2.5 fs solid 0.5 border lw 2 lc "red",\
     A using 1:2:(1) with hsteps below y=2.5 fs solid 0.5 border lw 2 lc "blue"

The resulting diagram is attached. This expression can't be obtained by the new patch (v2).

I think it is a natural association or extension of filledcurves or fillsteps styles.

set title "fillsteps"
plot A using 1:2 with fillsteps above y=2.5 fs solid 0.5 border lw 2 lc "red",\
     A using 1:2 with fillsteps below y=2.5 fs solid 0.5 border lw 2 lc "blue",\
     A using 1:2 with points pt 7 lt black

set title "filledcurves"
plot A using 1:2 with filledcurves above y=2.5 fs solid 0.5 border lw 2 lc "red",\
     A using 1:2 with filledcurves below y=2.5 fs solid 0.5 border lw 2 lc "blue",\
     A using 1:2 with points pt 7 lt black

I do not think that above/below should affect the curve itself, only the fill area.

Are there any other styles in which the border behaves like this way?

I removed the 'split' keyword in patch v.2, and I wrote,

Why would one ever use this rather than "with boxes"?

It was implemented with the intention of being used with vsteps. But, in the new patch, the split option has been removed along with the removal of vsteps.

Sorry, but, there are some examples to describe I needed this option, although you might feel it is a slight difference. Please, check the following sample codes which show the different aspect of 'hsteps split' from 'boxes'.

First example

Box expression of unevenly spaced data.

$data <<EOD
# Rmin Count1 Count2
0.01 1.2e5  8.2e4
0.02 5.0e4  9.0e3
0.05 1.1e4  2.3e3
0.1  5.8e3  7.2e2
0.2  1.2e3  3.0e2
0.5  6.2e2  8.1e1
1.0  2.4e2  3.7e1
2.0  9.3e1  1.3e1
5.0  NaN    NaN
EOD

set log y
set log x
set xrange [0.005:10]
set yrange [1:1e6]
set xlabel "Radius (mm)"
set ylabel "Count"
set ytics format "10^{%T}"

unset key

plot $data using 1:2 with hsteps split forward lw 3

To draw this with boxes, more complex data preparation seem to be needed, because
the boxes style don't have forward or backward

Second example

"hsteps split" does not draw baseline as border.

$data <<EOD
# Category G1(%) G2(%)
A 30 70
B 20 80
C 40 60
D 25 75
E 60 40
F 50 50
EOD

set lmargin screen 0.25
set rmargin screen 0.75
set yrange [-100:100]
set xrange [-1:6] 
set xtics ("100" -100, "50" -50, "0" 0, "50" 50, "100" 100)
set ytics scale 0
unset key

set grid xtics

set title "histeps split"
plot $data using 0:2:(0.6):xtic(1) with hsteps split fill solid 0.5 border lw 3, \
     $data using 0:(-$3):(0.6)     with hsteps split fill solid 0.5 border lw 3

Here is boxes version.

set title "boxes"
plot $data using 0:2:(0.6):xtic(1) with boxes fill solid 0.5 border lw 3, \
     $data using 0:(-$3):(0.6)     with boxes fill solid 0.5 border lw 3

The baseline (y=0) is shared by two plots and the baseline of first plot is overrided by the baseline of second plot.

Answers to various questions:

• Until now the only plot style that recognizes above/below as keywords is "with filledcurves". That plot style produces only the fill areas. It does not draw a border. If you want a border you must draw it separately using "with lines". So it is not an exact precedent for "with hsteps fillstyle solid". I initially expected that "hsteps above/below" would only affect the fill area. Having it also affect the border is not terrible; but it's not what I expected.

• There are lots of cases where if you set a parameter to NaN it will produce a bad plot. Consider set yrange [0:NaN] or set boxwidth NaN. I think that adding special tests for something like that is not warranted. At some point you have to trust that the user will not "shoot themself in the foot" (a famous saying in the unix world).

• I may not understand your idea of a "baseline". Most plot styles do not have such a concept, at least not something separate from the plot border. What is the problem if "with hsteps nolink" does not use a baseline? That mode also does not have a fill area, so I would expect it to ignore above/below anyway.

• I think it is OK if if "above" or "below" must come before "y=<foo>". That is the natural way to say it in English.</foo>

Anything else?

I have gotten sidetracked trying to analyze this use-after-free error that is triggered by rendering the version 3 set of figures with the cairo terminals. The problem is probably in gp_cairo.c, but nothing in the full demo set triggers it so the new hsteps code must use some access pattern or sequence of calls that differs from the existing plot styles.