Try adding Github Problem Matchers to libc++ workflow. #146768
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The problem matchers are simple regex's that will be applied to the output of particular steps, and when a match is found, an annotation is applied to the relevent github workflow. The idea is to more easily surface relevent failure information, which may otherwise be hidden among cancelled tasks. The Problem Matcher specification/format is rather limited in its descriptive abliity, meaning we lose most context about the error. Further, because many of the source files are re-arranged before use, it prevents us from actually pointing to the file:line in the github UI; Boo :-(. A more complete solution would likely hook into LIT, remap source file paths as needed, and use the "echo ::error::" toolkit command to create the annotations.
llvmbot
added
libc++
|
@llvm/pr-subscribers-libcxx Author: Eric (EricWF) ChangesThe problem matchers are simple regex's that will be applied to the output of particular steps, and when a match is found, an annotation is applied to the relevent github workflow. The idea is to more easily surface relevent failure information, which may otherwise be hidden among cancelled tasks. The Problem Matcher specification/format is rather limited in its descriptive abliity, meaning we lose most context about the error. Further, because many of the source files are re-arranged before use, it prevents us from actually pointing to the file:line in the github UI; Boo :-(. A more complete solution would likely hook into LIT, remap source file paths as needed, and use the "echo ::error::" toolkit command to create the annotations. Full diff: https://github.com/llvm/llvm-project/pull/146768.diff 2 Files Affected:
diff --git a/.github/libcxx-problem-matchers.json b/.github/libcxx-problem-matchers.json
new file mode 100644
index 0000000000000..d681b83f069a1
--- /dev/null
+++ b/.github/libcxx-problem-matchers.json
@@ -0,0 +1,27 @@
+{
+ "problemMatcher": [
+ {
+ "owner": "clang",
+ "pattern": [
+ {
+ "regexp": "^(.+):(\\d+):(\\d+):\\s+(error|warning):\\s+(.+)$",
+ "file": 1,
+ "line": 2,
+ "column": 3,
+ "severity": 4,
+ "message": 5
+ }
+ ]
+ },
+ {
+ "owner": "lit",
+ "severity": "error",
+ "pattern": [
+ {
+ "regexp": "^\\s*(FAIL|XPASS):\\s+([^\\s]+)\\s+::\\s+(.+?)\\s+\\(\\d+\\s+of\\s+\\d+\\)$",
+ "message": 2
+ }
+ ]
+ }
+ ]
+}
diff --git a/.github/workflows/libcxx-build-and-test.yaml b/.github/workflows/libcxx-build-and-test.yaml
index f0bdf6c0b5899..0145fe27d7563 100644
--- a/.github/workflows/libcxx-build-and-test.yaml
+++ b/.github/workflows/libcxx-build-and-test.yaml
@@ -57,10 +57,17 @@ jobs:
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: ${{ matrix.config }}.${{ matrix.cxx }}
- run: libcxx/utils/ci/run-buildbot ${{ matrix.config }}
+ run: |
+ echo "::add-matcher::.github/libcxx-problem-matchers.json"
+ libcxx/utils/ci/run-buildbot ${{ matrix.config }}
env:
CC: ${{ matrix.cc }}
CXX: ${{ matrix.cxx }}
+ - if: always()
+ name: Disable Problem Matchers
+ run: |
+ echo "::remove-matcher owner=lit::"
+ echo "::remove-matcher owner=clang::"
- uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
if: always()
with:
@@ -103,10 +110,17 @@ jobs:
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: ${{ matrix.config }}
- run: libcxx/utils/ci/run-buildbot ${{ matrix.config }}
+ run: |
+ echo "::add-matcher::.github/libcxx-problem-matchers.json"
+ libcxx/utils/ci/run-buildbot ${{ matrix.config }}
env:
CC: ${{ matrix.cc }}
CXX: ${{ matrix.cxx }}
+ - if: always()
+ name: Disable Problem Matchers
+ run: |
+ echo "::remove-matcher owner=lit::"
+ echo "::remove-matcher owner=clang::"
- uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
if: always() # Upload artifacts even if the build or test suite fails
with:
@@ -167,10 +181,17 @@ jobs:
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: ${{ matrix.config }}
- run: libcxx/utils/ci/run-buildbot ${{ matrix.config }}
+ run: |
+ echo "::add-matcher::.github/libcxx-problem-matchers.json"
+ libcxx/utils/ci/run-buildbot ${{ matrix.config }}
env:
CC: clang-21
CXX: clang++-21
+ - if: always()
+ name: Disable Problem Matchers
+ run: |
+ echo "::remove-matcher owner=lit::"
+ echo "::remove-matcher owner=clang::"
- uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
if: always()
with:
@@ -221,6 +242,7 @@ jobs:
- uses: seanmiddleditch/gha-setup-ninja@3b1f8f94a2f8254bd26914c4ab9474d4f0015f67 # v6
- name: Build and test
run: |
+ echo "::add-matcher::.github/libcxx-problem-matchers.json"
python3 -m venv .venv
source .venv/bin/activate
python -m pip install psutil
|
|
@llvm/pr-subscribers-github-workflow Author: Eric (EricWF) ChangesThe problem matchers are simple regex's that will be applied to the output of particular steps, and when a match is found, an annotation is applied to the relevent github workflow. The idea is to more easily surface relevent failure information, which may otherwise be hidden among cancelled tasks. The Problem Matcher specification/format is rather limited in its descriptive abliity, meaning we lose most context about the error. Further, because many of the source files are re-arranged before use, it prevents us from actually pointing to the file:line in the github UI; Boo :-(. A more complete solution would likely hook into LIT, remap source file paths as needed, and use the "echo ::error::" toolkit command to create the annotations. Full diff: https://github.com/llvm/llvm-project/pull/146768.diff 2 Files Affected:
diff --git a/.github/libcxx-problem-matchers.json b/.github/libcxx-problem-matchers.json
new file mode 100644
index 0000000000000..d681b83f069a1
--- /dev/null
+++ b/.github/libcxx-problem-matchers.json
@@ -0,0 +1,27 @@
+{
+ "problemMatcher": [
+ {
+ "owner": "clang",
+ "pattern": [
+ {
+ "regexp": "^(.+):(\\d+):(\\d+):\\s+(error|warning):\\s+(.+)$",
+ "file": 1,
+ "line": 2,
+ "column": 3,
+ "severity": 4,
+ "message": 5
+ }
+ ]
+ },
+ {
+ "owner": "lit",
+ "severity": "error",
+ "pattern": [
+ {
+ "regexp": "^\\s*(FAIL|XPASS):\\s+([^\\s]+)\\s+::\\s+(.+?)\\s+\\(\\d+\\s+of\\s+\\d+\\)$",
+ "message": 2
+ }
+ ]
+ }
+ ]
+}
diff --git a/.github/workflows/libcxx-build-and-test.yaml b/.github/workflows/libcxx-build-and-test.yaml
index f0bdf6c0b5899..0145fe27d7563 100644
--- a/.github/workflows/libcxx-build-and-test.yaml
+++ b/.github/workflows/libcxx-build-and-test.yaml
@@ -57,10 +57,17 @@ jobs:
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: ${{ matrix.config }}.${{ matrix.cxx }}
- run: libcxx/utils/ci/run-buildbot ${{ matrix.config }}
+ run: |
+ echo "::add-matcher::.github/libcxx-problem-matchers.json"
+ libcxx/utils/ci/run-buildbot ${{ matrix.config }}
env:
CC: ${{ matrix.cc }}
CXX: ${{ matrix.cxx }}
+ - if: always()
+ name: Disable Problem Matchers
+ run: |
+ echo "::remove-matcher owner=lit::"
+ echo "::remove-matcher owner=clang::"
- uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
if: always()
with:
@@ -103,10 +110,17 @@ jobs:
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: ${{ matrix.config }}
- run: libcxx/utils/ci/run-buildbot ${{ matrix.config }}
+ run: |
+ echo "::add-matcher::.github/libcxx-problem-matchers.json"
+ libcxx/utils/ci/run-buildbot ${{ matrix.config }}
env:
CC: ${{ matrix.cc }}
CXX: ${{ matrix.cxx }}
+ - if: always()
+ name: Disable Problem Matchers
+ run: |
+ echo "::remove-matcher owner=lit::"
+ echo "::remove-matcher owner=clang::"
- uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
if: always() # Upload artifacts even if the build or test suite fails
with:
@@ -167,10 +181,17 @@ jobs:
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: ${{ matrix.config }}
- run: libcxx/utils/ci/run-buildbot ${{ matrix.config }}
+ run: |
+ echo "::add-matcher::.github/libcxx-problem-matchers.json"
+ libcxx/utils/ci/run-buildbot ${{ matrix.config }}
env:
CC: clang-21
CXX: clang++-21
+ - if: always()
+ name: Disable Problem Matchers
+ run: |
+ echo "::remove-matcher owner=lit::"
+ echo "::remove-matcher owner=clang::"
- uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 # v4.3.0
if: always()
with:
@@ -221,6 +242,7 @@ jobs:
- uses: seanmiddleditch/gha-setup-ninja@3b1f8f94a2f8254bd26914c4ab9474d4f0015f67 # v6
- name: Build and test
run: |
+ echo "::add-matcher::.github/libcxx-problem-matchers.json"
python3 -m venv .venv
source .venv/bin/activate
python -m pip install psutil
|
You can test this locally with the following command:git-clang-format --diff HEAD~1 HEAD --extensions cpp -- libcxx/test/broken-on-purpose-1.pass.cppView the diff from clang-format here.diff --git a/libcxx/test/broken-on-purpose-1.pass.cpp b/libcxx/test/broken-on-purpose-1.pass.cpp
index ff8100c80..40da463af 100644
--- a/libcxx/test/broken-on-purpose-1.pass.cpp
+++ b/libcxx/test/broken-on-purpose-1.pass.cpp
@@ -1,4 +1,2 @@
-DoesNotExist bar() {
-
-}
+DoesNotExist bar() {}
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The problem matchers are simple regex's that will be applied to the output of particular steps, and when a match is found, an annotation is applied to the relevent github workflow.
The idea is to more easily surface relevent failure information, which may otherwise be hidden among cancelled tasks.
The Problem Matcher specification/format is rather limited in its descriptive abliity, meaning we lose most context about the error. Further, because many of the source files are re-arranged before use, it prevents us from actually pointing to the file:line in the github UI; Boo :-(.
A more complete solution would likely hook into LIT, remap source file paths as needed, and use the "echo ::error::" toolkit command to create the annotations.