Send Varnish logs to Honeycomb.io #12
Conversation
gerhard
force-pushed
the
send-logs-to-honeycomb
branch
4 times, most recently
from
a2fc9e5 to
c33b4dc
Compare
gerhard
force-pushed
the
send-logs-to-honeycomb
branch
6 times, most recently
from
b4e57b5 to
0366b12
Compare
gerhard
force-pushed
the
send-logs-to-honeycomb
branch
from
0366b12 to
401cab2
Compare
Builds on top of #12 Signed-off-by: Gerhard Lazu <[email protected]>
Builds on top of #12 This also changes the `debug` just command to require `team` access so that we can experiment with the FULL behaviour locally. Signed-off-by: Gerhard Lazu <[email protected]>
Builds on top of #12 This also changes the `debug` just command to require `team` access so that we can experiment with the FULL behaviour locally. Signed-off-by: Gerhard Lazu <[email protected]>
dagger/main.go
Outdated
| File("/go/bin/goreman") | ||
|
|
||
| procfile := fmt.Sprintf(`pipely: docker-varnish-entrypoint | ||
| vectorArchive := dag.HTTP("https://packages.timber.io/vector/" + vectorVersion + "/vector-" + vectorVersion + "-" + altArchitecture(ctx) + "-unknown-linux-gnu.tar.gz") |
There was a problem hiding this comment.
Would it be wise to check the SHA? Maybe commit vector.{vectorVersion}.{altArchitecture(ctx)}.sha to the repo and verify that they match.
There was a problem hiding this comment.
Yes, it most definitely would, especially for a core component such as Vector.
On it 🧑🏭
There was a problem hiding this comment.
So it seems that Vector does not publish sha digests: https://packages.timber.io/vector/0.47.0
Having looked at the official installer script, this one doesn't check digests or pin either: https://vector.dev/docs/setup/installation/manual/vector-installer/
Instead of adding something custom on top of the official install docs (I was thinking of manually computing the sha256 digest & baking it in), I went down the container path, pinned to a sha256, and copied all the necessary Vector paths out of it:
| platform := platforms.MustParse(string(p)) | ||
| // https://github.com/hatoo/oha/releases | ||
| oha := dag.HTTP("https://github.com/hatoo/oha/releases/download/v1.8.0/oha-linux-" + platform.Architecture) | ||
| oha := dag.HTTP("https://github.com/hatoo/oha/releases/download/v" + ohaVersion + "/oha-linux-" + platform.Architecture) |
There was a problem hiding this comment.
Another dynamically resolved resource that might be good to pin to a hash.
There was a problem hiding this comment.
Indeed! Since this is a debugging component, meaning that we only use it locally & never ship it, I am less concerned about it. Nice to have, for sure, but OK as is for now.
gerhard
force-pushed
the
send-logs-to-honeycomb
branch
from
401cab2 to
b49d5b4
Compare
Uses vector.dev which runs as a "sidecar" & ingests logs via `STDIN`. Varnish sends logs via a `varnishncsa` wrapper. It can be used locally, in debug mode, via: ``` varnish-json-response | jq . ```` One complication was figuring out how to extract the client IP correctly. While `fly-client-ip` request header is a Fly.io convenience, we need to support x-forwarded-for just in case the convenience header stops working, and for when we roll out the second provider - see: - #9 Other changes: - Removed `EDGE` from `cache-status` header - we don't have origin shields, and until we do (**if** we do) we don't want to "carry" this - Include `region` info in `/health` responses (otherwise it will be hard to tell from these events which regions are unhealthy) - Extracted a few `just` file fragments so that syntax highlighting would work in Zed 🤦♂️ - Updated all deps to latest stable NEXT: enrich logs with GeoIP data. Signed-off-by: Gerhard Lazu <[email protected]>
gerhard
force-pushed
the
send-logs-to-honeycomb
branch
from
b49d5b4 to
af7f790
Compare
Uses vector.dev which runs as a "sidecar" & ingests logs via
STDIN.Varnish sends logs via a
varnishncsawrapper. It can be used locally, in debug mode, via:One complication was figuring out how to extract the client IP correctly. While
fly-client-iprequest header is a Fly.io convenience, we need to support x-forwarded-for just in case the convenience header stops working, and for when we roll out the second provider - see:Other changes:
EDGEfromcache-statusheader - we don't have origin shields, and until we do (if we do) we don't want to "carry" thisregioninfo in/healthresponses (otherwise it will be hard to tell from these events which regions are unhealthy)justfile fragments so that syntax highlighting would work in Zed 🤦♂️NEXT: enrich logs with GeoIP data.
What properties do we capture for each event?
Do the existing boards work? No, but we have newer & cleaner boards that serve the same purpose:
Pipely SERVICE
Pipely REQUESTS
Also: Pipely CONTENT
TODOs
goremansupervision)stdinis simplest)pipely-gerhard-devdatasetpipelydatasetpipelydataset