Closed
Description
During a series of about a dozen AutonomousDatabase create/delete cycles, the CR remained stuck 3 times at either the Provisioning or the Terminating state even after the OCI ADB transitioned to Available or Terminated. This was due to a transient 500 error.
- It appears that there was no attempt to recover. The logs show:
DEBUG 2024/06/14 04:14:56.145788 asm_amd64.s:1650: Retry policy to use: {MaximumNumberAttempts=1, MinSleepBetween=0, MaxSleepBetween=0, ExponentialBackoffBase=0, NonEventuallyConsistentPolicy=<nil>}
- There was no indication on the CR (e.g., a condition) to show the error.
Here's the error:
INFO 2024/06/14 04:14:57.577023 client.go:463: Dump Response HTTP/1.1 500 Internal Server Error^M
Connection: close^M
Content-Length: 502^M
Cache-Control: must-revalidate,no-cache,no-store^M
Content-Type: text/html;charset=iso-8859-1^M
Date: Fri, 14 Jun 2024 04:14:56 GMT^M
Opc-Request-Id: 527875f6252c7e89ba40f484d3b47ef9/8077408D57D9A0CABBEC59A796F17A7F/C0B75A80CCCA38741AFB0B4820DD1D99^M
Strict-Transport-Security: max-age=31536000; includeSubDomains;^M
X-Content-Type-Options: nosniff^M
^M
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 Request failed.</title>
</head>
<body><h2>HTTP ERROR 500 Request failed.</h2>
<table>
<tr><th>URI:</th><td>/20160918/autonomousDatabases/ocid1.autonomousdatabase.oc1.iad.anuwcljs3lzhqhyadm6k5rdkjelgvsbp3k3c4ym4jqaivgemcsp6mgoqgqeq</td></tr>
<tr><th>STATUS:</th><td>500</td></tr>
<tr><th>MESSAGE:</th><td>Request failed.</td></tr>
<tr><th>SERVLET:</th><td>jersey</td></tr>
</table>
</body>
</html>
DEBUG 2024/06/14 04:14:57.577078 client.go:465: Error response could not be parsed due to: invalid character '<' looking for beginning of value
2024-06-14T04:14:57Z ERROR controllers.database.AutonomousDatabase.validateOperation.manageError UpdateFailed {"Namespace/Name": {"name":"oraclepdb-152abed7-367a-4631-b94e-156cf7b3ae59-adb","namespace":"test"}, "error": "[Error returned by Database Service. Http Status Code: 500. Error Code: BadErrorResponse. Opc request id: c35ed8b3f385c5d8191cfce81e1e6ee0/27F2B9A32C934160A70414F24047B67E/F7C056F133118AEEBC3207A1A6CD87FA. Message: Failed to parse json from response body due to: invalid character '<' looking for beginning of value. With response body <html>\n<head>\n<meta http-equiv=\"Content-Type\" content=\"text/html;charset=ISO-8859-1\"/>\n<title>Error 500 Request failed.</title>\n</head>\n<body><h2>HTTP ERROR 500 Request failed.</h2>\n<table>\n<tr><th>URI:</th><td>/20160918/autonomousDatabases/ocid1.autonomousdatabase.oc1.iad.anuwcljs3lzhqhyadm6k5rdkjelgvsbp3k3c4ym4jqaivgemcsp6mgoqgqeq</td></tr>\n<tr><th>STATUS:</th><td>500</td></tr>\n<tr><th>MESSAGE:</th><td>Request failed.</td></tr>\n<tr><th>SERVLET:</th><td>jersey</td></tr>\n</table>\n\n</body>\n</html>\n.\nOperation Name: GetAutonomousDatabase\nTimestamp: 2024-06-14 04:14:54 +0000 GMT\nClient Version: Oracle-GoSDK/65.49.3\nRequest Endpoint: GET https://database.us-ashburn-1.oraclecloud.com/20160918/autonomousDatabases/ocid1.autonomousdatabase.oc1.iad.anuwcljs3lzhqhyadm6k5rdkjelgvsbp3k3c4ym4jqaivgemcsp6mgoqgqeq
Metadata
Metadata
Assignees
Labels
No labels