Skip to content

feat: update grading workflows #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 29, 2025
Merged

feat: update grading workflows #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
64d32ca
refactor: simplify the check step process
FidelusAleksander Apr 17, 2025
59188f7
fix: correct markdown template paths and adjust workflow dependencies
FidelusAleksander Apr 17, 2025
6d0c48e
refactor: use yaml styling for template-vars
FidelusAleksander Apr 17, 2025
86eac3a
cleanup and review comments fix
FidelusAleksander Apr 18, 2025
d83f053
Use v2 of action-text-variables
FidelusAleksander Apr 22, 2025
24b9d26
refactor: use keyphrase-checker action
FidelusAleksander Apr 24, 2025
06c6fd3
add comments with todos
FidelusAleksander Apr 25, 2025
a1be7ae
Update step 2 grading workflow
FidelusAleksander Apr 25, 2025
8e75721
feat: use exercise-toolkit for markdown templates
FidelusAleksander Apr 28, 2025
c652aee
feat: update grading workflow in step 4b
FidelusAleksander Apr 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 7 additions & 12 deletions .github/workflows/0-start-exercise.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,22 +38,18 @@ jobs:
- name: Get response templates
uses: actions/checkout@v4
with:
repository: skills/response-templates
path: skills-response-templates

- name: Configure Git user
run: |
git config user.name github-actions[bot]
git config user.email github-actions[bot]@users.noreply.github.com
repository: skills/exercise-toolkit
path: exercise-toolkit
ref: v0.3.0

- name: Build comment - add step content
id: build-comment
uses: skills/action-text-variables@v1
uses: skills/action-text-variables@v2
with:
template-file: ${{ env.STEP_1_FILE }}
template-vars: |
login=${{ github.actor }}
full_repo_name=${{ github.repository }}
login: ${{ github.actor }}
full_repo_name: ${{ github.repository }}

- name: Create comment - add step content
run: |
Expand All @@ -66,13 +62,12 @@ jobs:
- name: Create comment - watching for progress
run: |
gh issue comment "$ISSUE_URL" \
--body-file "skills-response-templates/step-feedback/watching-for-progress.md"
--body-file "exercise-toolkit/markdown-templates/step-feedback/watching-for-progress.md"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Disable current workflow and enable next one
run: |
# gh workflow enable "Step 0" # Already disabled
gh workflow enable "Step 1"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
20 changes: 11 additions & 9 deletions .github/workflows/1-preparing.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,14 @@ jobs:
- name: Get response templates
uses: actions/checkout@v4
with:
repository: skills/response-templates
path: skills-response-templates
repository: skills/exercise-toolkit
path: exercise-toolkit
ref: main # TODO: Change later

- name: Update comment - checking work
run: |
gh issue comment "$ISSUE_URL" \
--body-file skills-response-templates/step-feedback/checking-work.md \
--body-file exercise-toolkit/markdown-templates/step-feedback/checking-work.md \
--edit-last
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -52,11 +53,11 @@ jobs:

- name: Build message - step finished
id: build-message-step-finish
uses: skills/action-text-variables@v1
uses: skills/action-text-variables@v2
with:
template-file: skills-response-templates/step-feedback/step-finished-prepare-next-step.md
template-file: exercise-toolkit/markdown-templates/step-feedback/step-finished-prepare-next-step.md
template-vars: |
next_step_number=2
next_step_number: 2

- name: Update comment - step finished
run: |
Expand All @@ -81,8 +82,9 @@ jobs:
- name: Get response templates
uses: actions/checkout@v4
with:
repository: skills/response-templates
path: skills-response-templates
repository: skills/exercise-toolkit
path: exercise-toolkit
ref: main # TODO: Change later

- name: Create comment - add step content
run: |
Expand All @@ -94,7 +96,7 @@ jobs:
- name: Create comment - watching for progress
run: |
gh issue comment "$ISSUE_URL" \
--body-file skills-response-templates/step-feedback/watching-for-progress.md
--body-file exercise-toolkit/markdown-templates/step-feedback/watching-for-progress.md
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Expand Down
84 changes: 39 additions & 45 deletions .github/workflows/2-first-introduction.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,74 +34,67 @@ jobs:
- name: Get response templates
uses: actions/checkout@v4
with:
repository: skills/response-templates
path: skills-response-templates
repository: skills/exercise-toolkit
path: exercise-toolkit
ref: main # TODO: Change later

- name: Update comment - checking work
run: |
gh issue comment "$ISSUE_URL" \
--body-file skills-response-templates/step-feedback/checking-work.md \
--body-file exercise-toolkit/markdown-templates/step-feedback/checking-work.md \
--edit-last
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

# START: Check practical exercise

# Search for the comment about registration validation
- name: Check contents of 'src/app.py'
run: |
# File and expected phrase
file="src/app.py"
keyphrase="Validate student is not already signed up"

# Fail the workflow if the file content is missing
if ! grep -q "$keyphrase" "$file"; then
message="It seems our registration validation bug has not been fixed. Please try again."
gh issue comment "$ISSUE_URL" \
--body "$message" \
--edit-last
exit 1
fi
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

# Check the number of activities in the file by counting the json keys
- name: Check for additional student activities
id: check-additional-activities
continue-on-error: true
uses: skills/action-keyphrase-checker@main # TODO: Change later
with:
text-file: src/app.py
keyphrase: '"description"'
minimum-occurrences: 3
case-sensitive: false

- name: Build message - step results
id: build-message-step-results
uses: skills/action-text-variables@v2
with:
template-file: exercise-toolkit/markdown-templates/step-feedback/step-results-table.md
template-vars: |
step_number: 2
passed: ${{ !contains(steps.*.outcome, 'failure') }}
results_table:
- description: "New activities added to src/app.py. We found ${{ steps.check-additional-activities.outputs.occurrences }} activities (minimum 4 required)"
passed: ${{ steps.check-additional-activities.outcome == 'success' }}

- name: Create comment - step results
run: |
# File and phrase to count
file="src/app.py"
keyphrase='"description":'
minimum_occurences=4

# Get the number of occurences of the keyphrase
found_occurences=$(grep -o "$keyphrase" "$file" | wc -l)

# If the number of occurences is less than the minimum, fail the workflow and send a message
if [ "$found_occurences" -lt "$minimum_occurences" ]; then
message="No new student activities were found. Please use Copilot to generate some and try again."
gh issue comment "$ISSUE_URL" \
--body "$message" \
--edit-last
exit 1
fi
gh issue comment "$ISSUE_URL" \
--body "$COMMENT_BODY" \
--edit-last
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COMMENT_BODY: ${{ steps.build-message-step-results.outputs.updated-text }}

# END: Check practical exercise
- name: Fail job if not all checks passed
if: contains(steps.*.outcome, 'failure')
run: exit 1

- name: Build message - step finished
id: build-message-step-finish
uses: skills/action-text-variables@v1
with:
template-file: skills-response-templates/step-feedback/step-finished-prepare-next-step.md
template-file: exercise-toolkit/markdown-templates/step-feedback/step-finished-prepare-next-step.md
template-vars: |
next_step_number=3

- name: Update comment - step finished
run: |
gh issue comment "$ISSUE_URL" \
--body "$ISSUE_BODY" \
--edit-last
--body "$ISSUE_BODY"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ISSUE_BODY: ${{ steps.build-message-step-finish.outputs.updated-text }}
Expand All @@ -120,8 +113,9 @@ jobs:
- name: Get response templates
uses: actions/checkout@v4
with:
repository: skills/response-templates
path: skills-response-templates
repository: skills/exercise-toolkit
path: exercise-toolkit
ref: main # TODO: Change later

- name: Create comment - add step content
run: |
Expand All @@ -133,7 +127,7 @@ jobs:
- name: Create comment - watching for progress
run: |
gh issue comment "$ISSUE_URL" \
--body-file skills-response-templates/step-feedback/watching-for-progress.md
--body-file exercise-toolkit/markdown-templates/step-feedback/watching-for-progress.md
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Expand Down
113 changes: 38 additions & 75 deletions .github/workflows/3-copilot-edits.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,86 +34,50 @@ jobs:
- name: Get response templates
uses: actions/checkout@v4
with:
repository: skills/response-templates
path: skills-response-templates
repository: skills/exercise-toolkit
path: exercise-toolkit
ref: main # TODO: Change later

- name: Update comment - checking work
run: |
gh issue comment "$ISSUE_URL" \
--body-file skills-response-templates/step-feedback/checking-work.md \
--body-file exercise-toolkit/markdown-templates/step-feedback/checking-work.md \
--edit-last
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

# START: Check practical exercise

- name: Check participant info on activity cards
id: check-user-work
run: |
# Checks to perform
checks='{
"app_js": {
"name": "app.js",
"passed": true,
"message": ""
},
"styles_css": {
"name": "styles.css",
"passed": true,
"message": ""
}
}'

# Check for participants info in app.js
file="src/static/app.js"
keyphrase='participant'
minimum_occurences=3
found_occurences=$(grep -o "$keyphrase" "$file" | wc -l)
if [ "$found_occurences" -lt "$minimum_occurences" ]; then
checks=$(echo $checks | jq '.app_js.passed = false')
checks=$(echo $checks | jq '.app_js.message = "Please use Copilot to update the web application activity cards to show participant info."')
fi

# Check for participants info in styles.css
file="src/static/styles.css"
keyphrase='participant'
minimum_occurences=1
found_occurences=$(grep -o "$keyphrase" "$file" | wc -l)
if [ "$found_occurences" -lt "$minimum_occurences" ]; then
checks=$(echo $checks | jq '.styles_css.passed = false')
checks=$(echo $checks | jq '.styles_css.message = "Please use Copilot to update the web application styling to support participant info."')
fi

# Verify all checks passed
passed=$(echo $checks | jq '. | all(.passed?)')

# Flatten to an array for returning. Allows iteration during rendering.
results=$(echo $checks | jq 'to_entries | map({name: .key} + .value)')

# Save pass status to output
echo "passed=$passed" >> $GITHUB_OUTPUT

# Save results to output
echo 'results<<EOF' >> $GITHUB_OUTPUT
echo $results >> $GITHUB_OUTPUT
echo 'EOF' >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Check for participant info in app.js
id: check-app-js
continue-on-error: true
uses: skills/action-keyphrase-checker@main # TODO: Change later
with:
text-file: src/static/app.js
keyphrase: participant
minimum-occurrences: 3
case-sensitive: false

- name: Check for participant info in styles.css
id: check-styles
continue-on-error: true
uses: skills/action-keyphrase-checker@main # TODO: Change later
with:
text-file: src/static/styles.css
keyphrase: participant
case-sensitive: false

- name: Build message - step results
id: build-message-step-results
uses: skills/action-text-variables@v1
uses: skills/action-text-variables@v2
with:
template-file: skills-response-templates/step-feedback/step-results.md
template-vars: '{
"step_number": 3,
"passed": ${{ steps.check-user-work.outputs.passed }},
"results_table": ${{ steps.check-user-work.outputs.results }},
"tips": [
"Copilot is becoming more capable everyday. Make sure to always be experimenting!",
"Try testing what Copilot can do. Something like: Add a dark mode toggle in the top right."
]
}'
template-file: exercise-toolkit/markdown-templates/step-feedback/step-results-table.md
template-vars: |
step_number: 3
passed: ${{ !contains(steps.*.outcome, 'failure') }}
results_table:
- description: "Check app.js for participant info"
passed: ${{ steps.check-app-js.outcome == 'success' }}
- description: "Participant styling updated in styles.css"
passed: ${{ steps.check-styles.outcome == 'success' }}

- name: Create comment - step results
run: |
Expand All @@ -125,16 +89,14 @@ jobs:
COMMENT_BODY: ${{ steps.build-message-step-results.outputs.updated-text }}

- name: Fail job if not all checks passed
if: steps.check-user-work.outputs.passed == 'false'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to stay to fail the grading job. I think the if should switch to ${{ !contains(steps.*.outcome, 'failure') }}

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check should be without the ! negation though, ${{ contains(steps.*.outcome, 'failure') }}

if: contains(steps.*.outcome, 'failure')
run: exit 1

# END: Check practical exercise

- name: Build message - step finished
id: build-message-step-finish
uses: skills/action-text-variables@v1
with:
template-file: skills-response-templates/step-feedback/step-finished-prepare-next-step.md
template-file: exercise-toolkit/markdown-templates/step-feedback/step-finished-prepare-next-step.md
template-vars: |
next_step_number=4

Expand All @@ -160,8 +122,9 @@ jobs:
- name: Get response templates
uses: actions/checkout@v4
with:
repository: skills/response-templates
path: skills-response-templates
repository: skills/exercise-toolkit
path: exercise-toolkit
ref: main # TODO: Change later

- name: Create comment - add step content
run: |
Expand All @@ -173,7 +136,7 @@ jobs:
- name: Create comment - watching for progress
run: |
gh issue comment "$ISSUE_URL" \
--body-file skills-response-templates/step-feedback/watching-for-progress.md
--body-file exercise-toolkit/markdown-templates/step-feedback/watching-for-progress.md
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Expand Down
Loading
Loading