Update details around skb->tstamp and skb->tstamp_type #168
Description
The way the meaning of tstamp and tstamp_type changed over time is a bit confusing at first. Some sources:
We should update the __sk_buff page and the bpf_skb_set_tstamp helper function page. I reconstructed to following
So also note that this helper description seems to be out of date, given the changes in the last commit.
So as I understand, tstamp used to be only for receiving. Then the FQ EDT mechanism was added. Both operate in CLOCK_REALTIME (time since boot).
Because the usage of the field was now overloaded, they made it so it resets to 0 on redirect (as not to confuse receive time with a requested delivery time).Apparently userspace programs can also set the tstamp field with the SO_TXTIME socket option. This is combined with the ETF qdisc. See torvalds/linux@25db26a
So, now these two interact. When a packet is redirected it is reset to 0 but unlike FQ, the ETF qdisc interprets this as the packet already being expired and will drop it.
The tstamp_time was added to deconflict. By default it will be BPF_SKB_CLOCK_REALTIME /BPF_SKB_TSTAMP_UNSPEC and enjoy old behavior. However, when userspace uses the SO_TXTIME socket option, it may be set to BPF_SKB_CLOCK_MONOTONIC or BPF_SKB_CLOCK_TAI . The tstamp will only be reset on redirect for BPF_SKB_CLOCK_REALTIME /BPF_SKB_TSTAMP_UNSPEC the other clock types are not reset (to pass the userspace value to ETF).
The bpf_skb_set_tstamp helper essentially provides the same functionality as userspace has with the socket option.
As far as I am able to tell, FQ is not aware of these clock types, and will always interpret the tstamp as CLOCK_REALTIME . Only ETF can actually deal with CLOCK_TAI and CLOCK_MONOTONIC values. So it seems this helper is only useful when.
For backwards compatibility purposes, the kernel does a clever trick where it checks if your BPF program is aware of tstamp_type . If you program reads the field, the verifier assumes the program is aware of the different types of time, and will always return the actual value. If a program never looks at the tstamp_type then the program will see 0 if under the hood non realtime time is used. And the same for writing, if you never read tstamp_type then writing to tstamp will automatically set tstamp_type to BPF_SKB_CLOCK_REALTIME . Only when the program is aware of timestamp types will it not modify tstamp_type when tstamp is written to.
As of now, you can use bpf_skb_set_tstamp during your ingress program to set a delivery time, as long as you use a non-realtime clock type. However, remember that FQ will still read it as if it were a realtime. This would only work for v5.18 and up. Alternatively, you can set the timestamp in your egress program, this should work on both old and new kernels.