[Performance] Varnish user context hash vcl should use the hash in vcl_hash

[bastnic]

We use Varnish and user context hash on very high traffic website. And we experienced various performance and stability issue I wanted to share with you. All credits for the finding does not belong to me but to my client and his exchanges with Varnish support. I think it's important enough to share it here.

User context hash is based on a new header that we vary on. This header itself is calculated on the /_fos_user_context_hash call that itself is vary against the cookie or authorization header.

If we use user context hash it's because we want the cardinality of an user context page to be as low as posible to avoid pointless computation.
But the cardinality of the _fos_user_context_hash is linked to the number of visitor, and it can be very big. We relates on the Vary header to make the distinction and find in all the cached variants.

The vary header is like a sql where on a not indexed field, against a possible enormous table if you have a traffic like this one.
To fix it we just need to index this field.

sub vcl_hash {
    if (req.url == "/_fos_user_context_hash") {
        hash_data(req.http.<Cookie or Autorization>);
    }
    # VS: NOTE: No return, fallthrough to builtin VCL
}

Before on our load test:

After:

On production, we reduced our number of varnish servers by ten.

The comportment is now explain in the next release varnish documentation along with a warning. The warning is triggered if there is more than 10 variants. And I hope everybody using user context hash has more than 10 logged in users otherwise...

[dbu]

On production, we reduced our number of varnish servers by ten.

wow, that optimization is on a level of having some direct ecological and financial impact!

and clever idea, to hash on that instead of vary. how could we implement this in a robust and backwards-compatible way? is there a step in the VCL where we could strip the vary from the response, if we know that we hash the request on the credentials? (or actually, does the vary even hurt in that case? we will only ever have one variant per cache then). so is all that is needed that bit of hashing the request when it is a request for the context hash?

we could add a hash sub into https://github.com/FriendsOfSymfony/FOSHttpCache/blob/master/resources/config/varnish/fos_user_context_url.vcl (because the custom hashing depends on the url, and VCL does not have variables). to be consistent with our other VCL, it should be a custom sub that the user has to call explicitly from their vcl_hash.

when we provide a hashing vcl in FOSHttpCache , we should include the domain imho, to be robust when people have multiple domains behind the same varnish.

do you want to do a pull request for this?

---

btw, in the project where we currently use this, we use the payed varnish variant. we are using the http client plugin to make an actual web request to the user hash url and store a key-value map of credentials => hash because restarting the request is inelegant and inefficient. (it should be possible with the varnish curl plugin as well: https://foshttpcache.readthedocs.io/en/latest/varnish-configuration.html#user-context - but i never tried it so we don't have a working example for it)

[bastnic]

so is all that is needed that bit of hashing the request when it is a request for the context hash?

Yes. For what it's worth we have others endpoints that vary on the user itself and we also hash it in that case. Depending on the url because that suit our need.

do you want to do a pull request for this?

I'm pretty busy right now, so if you want to, feel free to take over.

btw, in the project where we currently use this, we use the payed varnish variant

yep, they have cool features. Maybe document do a benchmark to compare performance?