-
Notifications
You must be signed in to change notification settings - Fork 34
Implement GitHub Action to automate patches and documentation #343
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
* feat: Add automation for modifying Oracle patch manifests - Implemented a Python script (`modify_patches.py`) to read, process, and insert patch data into a target YAML file. - Created a GitHub Actions workflow (`modify_patches.yml`) to automate the execution of the patch modification script on pull requests to the master branch. - Added unit tests (`modify_patchlist_test.py`) to validate the functionality of the patch modification logic. - Included a requirements file (`requirements.txt`) for necessary Python dependencies. - Updated `.gitignore` to exclude Python cache files. - Added a sample input YAML file (`modify_patchlist.yml`) to serve as a template for patch data. - Documented the structure and purpose of the patch manifest in `modify_patchlist.md`. - Cleaned up existing patch entries in `roles/common/defaults/main.yml` to prepare for new entries. * feat: Add user guide for patch automation process * fix: Update file extension references from .yaml to .yml in documentation and scripts * Refactor the GCS download/transfer roles into a more logical and better flowing set of tasks * Refined quoting for ssh remote commands in Ansible tasks; Minor adjustments to GCS software provisioning tasks * Set delete_control_node to False for the presubmit test (google#320) * Set delete_control_node to False for the presubmit test. * increase loop timeout from 2 to 3 hours * Add `whenever sqlerror exit sql.sqlcode` to all sqlplus blocks * Set `db_create_file_dest` when not using ASM storage * Change from `whenever sqlerror exit sql.sqlcode` to `whenever sqlerror exit failure` * Configure RAC single-instance presubmit tests to use Oracle Multitenant To get a bit more test coverage, here we configure the RAC single-instance test case (which is the faster one anyway) to set `--ora-db-container` to true, setting up multitenant. * Fix multitenant check when saving PDB state (google#326) * Fix multitenant check when saving PDB state * Change `container_db` to a boolean * Set pwd_gen_cmd conditionally based on ora-db-container flag (google#327) * Set pwd_gen_cmd conditionally based on ora-db-container flag * remove explicit cast to bool * Remove `whenever sqlerror` when shutting down from mount; Use PL/SQL to enable flashback and force logging to avoid errors * Take `oracle_edition` into consideration when building the RDBMS software file list * Add `list` filter when building `rdbms_edition_sw` for Ansible 2.9 compatibility * Add new readiness check to ensure that specified storage parameter disk groups exist in `asm_disks` * Initial commit * Fix basic syntax errors * Refactor commit trap and log setup Modify the common lib to also set up the cleanup trap as part of it's init, as one less thing the callers might forget. And talking about forgetting, we had a setup_logging function, and never called it. Refactoring so that watch_logs does both the setup and watching. It's long, but there's really no need for callers to run part of it. * Refactor commit trap and log setup Modify the common lib to also set up the cleanup trap as part of it's init, as one less thing the callers might forget. And talking about forgetting, we had a setup_logging function, and never called it. Refactoring so that watch_logs does both the setup and watching. It's long, but there's really no need for callers to run part of it. * Initial commit * Fix basic syntax errors * Clean up merge artifacts * Update templateing logic Modify the sed logic to make replacements into a remplrary file and to use that, allowing the presubmimt to be run multiple times without clobbering itself. Also removing a mistaken commented-out apk, adding an error check, and sending error messages to stderr (&2). * Add ora_db_container and improve error checks Free edition runs Oracle 23c, which no longer supports non-CDBs. Using the default terraform `ora_db_container` value of false, installs fail. This change explicitly sets it to `false` for data guard and `true` for free edition tests. Also some improved error checking: * If the sourcing fails, bail out * If the infra manager apply command fails, run a describe command to try and get the actual Terraform error if possible * Add ora_db_container and improve error checks Free edition runs Oracle 23c, which no longer supports non-CDBs. Using the default terraform `ora_db_container` value of false, installs fail. This change explicitly sets it to `false` for data guard and `true` for free edition tests. Also some improved error checking: * If the sourcing fails, bail out * If the infra manager apply command fails, run a describe command to try and get the actual Terraform error if possible * Override and set `ORA_DB_CONTAINER=TRUE` for Free edition installs * set workload_agent_username conditionally * Fix DG setup issues related to the staticConnectidentifier value and listener.ora & tnsnames.ora entries * Use `ansible_hostname` instead of `ansible_fqdn` when building tnsnames.ora entry * Build and set proper Data Guard StaticConnectIdentifier string * Changes to support installing Free edition on EL9 with new `os_version` key-value in `rdbms_software` * Updated filters for building list of RDBMS software to make `os_version` optional * Make disks larger for free edition test Oracle 23c's DBCA requires at least 13GB of space on the RECO disk. Also extending the data disk because, until google#335 lands, the DATA and RECO disks are classified as ASM disks and ignored, so the full install actually goes on the boot disk. Separately, make the free edition script executable in case someone wants to run it directly using ./ * Address review comments Check if ${deployment_name} is set in setup_vars() Set `delete-confrol-node` to `false` to avoid a race condition on long-running presubmits. * Data Guard changes required to support deployments with XFS storage * Change tnsnames.ora entries from `LISTENER_{{ db_name }}` to `LISTENER_{{ oracle_sid }}` * Break `roles/common/defaults/main.yml` into multiple separate files for manageability * Changes to support optionally procuring software from a URL * Remove `url_username` and `url_password` for `uri` module tasks * Refactor output handling in modify_patches.py to create multiple YAML files for different patch types and update tests accordingly * fix: merge errors * fix: merge errors * fix: merge error * fix: merge error --------- Co-authored-by: Simon Pane <[email protected]> Co-authored-by: Simon Pane <[email protected]> Co-authored-by: AlexBasinov <[email protected]> Co-authored-by: Marc Fielding <[email protected]> Co-authored-by: Alexey Basinov <[email protected]>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Pequod55 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @Pequod55. Thanks for your PR. I'm waiting for a google member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Change Description:
This PR introduces a comprehensive automation system for managing Oracle software and patch definitions. It establishes a new workflow where developers can define new patches in a central YAML file, and a GitHub Action automatically updates the necessary configuration files and user documentation.
Solution Overview:
.github/workflows/modify_patches.yml): Triggers on pull requests tomasterand on manual dispatch. It orchestrates the entire automation process.modify_patchlist.yml): A newly structured YAML file where users can add new software and patch definitions. The existing example entries have been commented out to serve as a template.modify_patches.py: A new Python script that parsesmodify_patchlist.ymland inserts the new patch data into the primary Ansible configuration file (roles/common/defaults/main.yml).modify_documentation.py: A new Python script that similarly parses the input YAML and automatically updates the software download tables in the user guide (docs/user-guide.md).stefanzweifel/git-auto-commit-actionto commit the file changes generated by the scripts directly back to the pull request branch.pytestandpytest-mockfor testing, with dependencies listed in the newrequirements.txt.modify_patchlist_test.py, with a comprehensive integration test for themodify_patches.pyscript..pytest_cachedirectory has been included. It is recommended to add.pytest_cache/to the project's.gitignorefile.)modify_patchlist.md): A detailed user and technical guide has been added to explain the new automation workflow, file structure, and how to add new patches.Test Commands:
Test Prep:
modify_patchlist.yml, uncomment an example entry or add a new one. For example, add a new RDBMS patch:documentation_overridessection to test custom documentation text:Test 1: Run unit tests
Test 2: Run the automation scripts locally
roles/common/defaults/main.ymlanddocs/user-guide.md) to verify the changes.Test 3: Run the full Git workflow (End-to-End Test)
modify_patchlist.yml:git add modify_patchlist.yml git commit -m "feat: Add test RDBMS patch 21.99"masterto trigger the "Modify Patches" GitHub Action.Expected Results:
For Test 1 (Unit Tests):
pytestcommand completes successfully, with all tests passing.For Test 2 (Local Run):
docs/user-guide.mdis updated to include a new row for the test patch, using the custom text from thedocumentation_overridesif provided.roles/common/defaults/main.ymlis updated with the new21.99.0.0.0patch entry under therdbms_patcheslist.modify_patchlist.ymlremains unchanged.For Test 3 (PR Workflow):
roles/common/defaults/main.ymlanddocs/user-guide.md.modify_patchlist.ymlfile is not changed by the automation commit.