Add support for asynchronous execution.
authorEtsuro Fujita <[email protected]>
Wed, 31 Mar 2021 09:45:00 +0000 (18:45 +0900)
committerEtsuro Fujita <[email protected]>
Wed, 31 Mar 2021 09:45:00 +0000 (18:45 +0900)
commit27e1f14563cf982f1f4d71e21ef247866662a052
tree4e1a17a61abbfc67a471760cc84e1e46182bfb9d
parent66392d396508c91c2ec07a61568bf96acb663ad8
Add support for asynchronous execution.

This implements asynchronous execution, which runs multiple parts of a
non-parallel-aware Append concurrently rather than serially to improve
performance when possible.  Currently, the only node type that can be
run concurrently is a ForeignScan that is an immediate child of such an
Append.  In the case where such ForeignScans access data on different
remote servers, this would run those ForeignScans concurrently, and
overlap the remote operations to be performed simultaneously, so it'll
improve the performance especially when the operations involve
time-consuming ones such as remote join and remote aggregation.

We may extend this to other node types such as joins or aggregates over
ForeignScans in the future.

This also adds the support for postgres_fdw, which is enabled by the
table-level/server-level option "async_capable".  The default is false.

Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself.  This commit
is mostly based on the patch proposed by Robert Haas, but also uses
stuff from the patch proposed by Kyotaro Horiguchi and from the patch
proposed by Thomas Munro.  Reviewed by Kyotaro Horiguchi, Konstantin
Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and
others.

Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com
Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
39 files changed:
contrib/postgres_fdw/connection.c
contrib/postgres_fdw/expected/postgres_fdw.out
contrib/postgres_fdw/option.c
contrib/postgres_fdw/postgres_fdw.c
contrib/postgres_fdw/postgres_fdw.h
contrib/postgres_fdw/sql/postgres_fdw.sql
doc/src/sgml/config.sgml
doc/src/sgml/fdwhandler.sgml
doc/src/sgml/monitoring.sgml
doc/src/sgml/postgres-fdw.sgml
src/backend/commands/explain.c
src/backend/executor/Makefile
src/backend/executor/README
src/backend/executor/execAmi.c
src/backend/executor/execAsync.c [new file with mode: 0644]
src/backend/executor/nodeAppend.c
src/backend/executor/nodeForeignscan.c
src/backend/nodes/copyfuncs.c
src/backend/nodes/outfuncs.c
src/backend/nodes/readfuncs.c
src/backend/optimizer/path/costsize.c
src/backend/optimizer/plan/createplan.c
src/backend/postmaster/pgstat.c
src/backend/storage/ipc/latch.c
src/backend/utils/misc/guc.c
src/backend/utils/misc/postgresql.conf.sample
src/include/executor/execAsync.h [new file with mode: 0644]
src/include/executor/nodeAppend.h
src/include/executor/nodeForeignscan.h
src/include/foreign/fdwapi.h
src/include/nodes/execnodes.h
src/include/nodes/plannodes.h
src/include/optimizer/cost.h
src/include/pgstat.h
src/include/storage/latch.h
src/test/regress/expected/explain.out
src/test/regress/expected/incremental_sort.out
src/test/regress/expected/insert_conflict.out
src/test/regress/expected/sysviews.out