Noticed at #12256 (comment), even though we have a bunch of checks in place for commit message, stale PR branch, etc., we do not fail CI for merge commits in PR branches.
I see github action https://github.com/greenled/no-merge-commits-check, maybe we can re-use the script in this repo and create a CI task to fail build if merge commits exist, and provide a message with link to https://checkstyle.org/beginning_development.html#Starting_Development .