Skip to content

Automate Graceful Recovery NFR #1832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
May 3, 2024
Merged
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
175036b
Add https traffic support for automated tests
bjee19 Mar 29, 2024
f0f4156
Add NGF and NGINX container restart tests
bjee19 Apr 16, 2024
8150af1
Add restart count to check container restart
bjee19 Apr 17, 2024
e6b90f5
Move restart count checks to helper function
bjee19 Apr 17, 2024
a31141a
Add polling wait functions to replace sleep workarounds
bjee19 Apr 18, 2024
b3cf4a5
Add polling wait for leader election lease to change and refactored e…
bjee19 Apr 19, 2024
48b9cc1
Refactor container restart functions
bjee19 Apr 19, 2024
e530198
Refactor container restart command to use job
bjee19 Apr 25, 2024
1eeafd2
Refactor waitForRoutesToBeReady to use parentrefs instead of status
bjee19 Apr 25, 2024
801a187
Revert runAsNonRoot to true
bjee19 Apr 25, 2024
8134a94
Change test to allow system suite to deploy NGF
bjee19 Apr 25, 2024
2df9b80
Add propagation policy when deleting Job
bjee19 Apr 25, 2024
3f4d37c
Refactor multiple functions into restartContainer function
bjee19 Apr 26, 2024
24eec08
Refactor test cases using shared runTest function
bjee19 Apr 26, 2024
8ae6c60
Change runTest name to runRecoveryTest
bjee19 Apr 26, 2024
f47df82
Add noling:gosec to satisfy CodeQL
bjee19 Apr 29, 2024
020a0b8
Remove accidentally added files
bjee19 Apr 29, 2024
1f33460
Add placeholder values in node debugger job
bjee19 Apr 29, 2024
851de6b
Add check for job spec container count
bjee19 Apr 29, 2024
cf6bd61
Remove regex from pgrep commands
bjee19 Apr 29, 2024
afb3af9
Remove leader lease checking from nginx container restart
bjee19 Apr 29, 2024
1ad5271
Add ngfPodName to BeforeAll
bjee19 Apr 29, 2024
7eaad5b
Fix argument refactor mistake
bjee19 Apr 29, 2024
d513eeb
Add ContainerRestartTimeout
bjee19 Apr 30, 2024
b4c7628
Move batchv1 scheme to system suite test
bjee19 Apr 30, 2024
bbd7448
Refactor constants to local variables
bjee19 Apr 30, 2024
4544fb5
Add additional comments
bjee19 Apr 30, 2024
1c3ecee
Refactor error checking
bjee19 Apr 30, 2024
6508425
Remove sinceSeconds when checking container logs for errors
bjee19 Apr 30, 2024
cefead3
Refactor waiting functions using gomega Eventually assertions
bjee19 May 1, 2024
587b237
Add Skip of failing test
bjee19 May 1, 2024
dc07841
Refactored Eventually functions to return errors instead of booleans
bjee19 May 1, 2024
d311b9d
Add check for lease holder identity being empty
bjee19 May 1, 2024
dbba349
Add filter of error messages in nginx logs to allow expected errors
bjee19 May 2, 2024
b5dc80e
Add ubuntu image version
bjee19 May 3, 2024
e326e96
Add teardown and setup of NGF and check NGF pod length is one
bjee19 May 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add check for lease holder identity being empty
  • Loading branch information
bjee19 committed May 3, 2024
commit d311b9d1dcecb0945fcc05f6b6db74f67dbff4b2
9 changes: 7 additions & 2 deletions tests/suite/graceful_recovery_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -269,8 +269,8 @@ func checkLeaderLeaseChange(originalLeaseName string) error {
return err
}

if originalLeaseName != leaseName {
return fmt.Errorf("originalLeaseName: %s, does not match current leaseName: %s", originalLeaseName, leaseName)
if originalLeaseName == leaseName {
return fmt.Errorf("expected originalLeaseName: %s, to not match current leaseName: %s", originalLeaseName, leaseName)
}

return nil
Expand All @@ -286,6 +286,11 @@ func getLeaderElectionLeaseHolderName() (string, error) {
if err := k8sClient.Get(ctx, key, &lease); err != nil {
return "", errors.New("could not retrieve leader election lease")
}

if *lease.Spec.HolderIdentity == "" {
return "", errors.New("leader election lease holder identity is empty")
}

return *lease.Spec.HolderIdentity, nil
}

Expand Down