pgstat: Infrastructure for more detailed IO statistics
authorAndres Freund <[email protected]>
Thu, 9 Feb 2023 04:53:42 +0000 (20:53 -0800)
committerAndres Freund <[email protected]>
Thu, 9 Feb 2023 04:53:42 +0000 (20:53 -0800)
commit28e626bde00efe8051fbd677227c97b1aa3c6a1a
tree4f54973fcd7b65d71d5003f2385aafbda771c27b
parent49c2c5fcb1e1e4311f08687f78dd9121330ae637
pgstat: Infrastructure for more detailed IO statistics

This commit adds the infrastructure for more detailed IO statistics. The calls
to actually count IOs, a system view to access the new statistics,
documentation and tests will be added in subsequent commits, to make review
easier.

While we already had some IO statistics, e.g. in pg_stat_bgwriter and
pg_stat_database, they did not provide sufficient detail to understand what
the main sources of IO are, or whether configuration changes could avoid
IO. E.g., pg_stat_bgwriter.buffers_backend does contain the number of buffers
written out by a backend, but as that includes extending relations (always
done by backends) and writes triggered by the use of buffer access strategies,
it cannot easily be used to tune background writer or checkpointer. Similarly,
pg_stat_database.blks_read cannot easily be used to tune shared_buffers /
compute a cache hit ratio, as the use of buffer access strategies will often
prevent a large fraction of the read blocks to end up in shared_buffers.

The new IO statistics count IO operations (evict, extend, fsync, read, reuse,
and write), and are aggregated for each combination of backend type (backend,
autovacuum worker, bgwriter, etc), target object of the IO (relations, temp
relations) and context of the IO (normal, vacuum, bulkread, bulkwrite).

What is tracked in this series of patches, is sufficient to perform the
aforementioned analyses. Further details, e.g. tracking the number of buffer
hits, would make that even easier, but was left out for now, to keep the scope
of the already large patchset manageable.

Bumps PGSTAT_FILE_FORMAT_ID.

Author: Melanie Plageman <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Justin Pryzby <[email protected]>
Reviewed-by: Kyotaro Horiguchi <[email protected]>
Discussion: https://postgr.es/m/20200124195226[email protected]
15 files changed:
doc/src/sgml/monitoring.sgml
src/backend/utils/activity/Makefile
src/backend/utils/activity/meson.build
src/backend/utils/activity/pgstat.c
src/backend/utils/activity/pgstat_bgwriter.c
src/backend/utils/activity/pgstat_checkpointer.c
src/backend/utils/activity/pgstat_io.c [new file with mode: 0644]
src/backend/utils/activity/pgstat_relation.c
src/backend/utils/activity/pgstat_shmem.c
src/backend/utils/activity/pgstat_wal.c
src/backend/utils/adt/pgstatfuncs.c
src/include/miscadmin.h
src/include/pgstat.h
src/include/utils/pgstat_internal.h
src/tools/pgindent/typedefs.list