ui,api,server: template categorization based on os #10773

shwstppr · 2025-04-24T12:29:26Z

Description

Adds new interface for image selection (template/iso) for an instance in UI.
Old interface can still be used and it can be configured using UI configuration (config.json)

OS categories/Guest OS categories have been improved with ability to create new categories, delete an existing category, and marking a category as featured to allow it to show up in the UI in the image selection interface.

New APIs added:

addOsCategory
deleteOsCategory
updateOsCategory

APIs updated:

updateOsType
listTemplates
listOsCategories

Several improvements in UI especially related to forms - DeloyVM, ReinstallVM, CreateVnfAppliance, AddAutoscaleGroup.

DeployVM form can now be opened from template/ISO details view with query params.

Reorganized (removed and added some) OS categories to the following (in the same order):

1. Ubuntu
2. Debian
3. Fedora
4. CentOS
5. Rocky Linux
6. Alma Linux
7. Oracle
8. RedHat
9. SUSE
10. Windows
11. Other

Documentation PR: apache/cloudstack-documentation#500

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov · 2025-04-24T12:37:06Z

Codecov Report

Attention: Patch coverage is 56.76275% with 195 lines in your changes missing coverage. Please review.

Project coverage is 16.61%. Comparing base (7632814) to head (c5c2f37).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...in/java/com/cloud/server/ManagementServerImpl.java	67.70%	27 Missing and 4 partials ⚠️
.../command/admin/guest/DeleteGuestOsCategoryCmd.java	0.00%	21 Missing ⚠️
.../command/admin/guest/UpdateGuestOsCategoryCmd.java	32.25%	21 Missing ⚠️
...m/cloud/resource/icon/dao/ResourceIconDaoImpl.java	0.00%	19 Missing ⚠️
...api/command/admin/guest/AddGuestOsCategoryCmd.java	31.81%	15 Missing ⚠️
...main/java/com/cloud/storage/GuestOSCategoryVO.java	45.00%	11 Missing ⚠️
...ain/java/com/cloud/api/query/QueryManagerImpl.java	0.00%	11 Missing ⚠️
...ain/java/com/cloud/storage/dao/GuestOSDaoImpl.java	0.00%	9 Missing ⚠️
...om/cloud/resourceicon/ResourceIconManagerImpl.java	0.00%	8 Missing ⚠️
...c/main/java/com/cloud/utils/db/GenericDaoBase.java	0.00%	7 Missing ⚠️
... and 12 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #10773      +/-   ##
============================================
+ Coverage     16.57%   16.61%   +0.03%     
- Complexity    13868    13927      +59     
============================================
  Files          5719     5729      +10     
  Lines        507178   508070     +892     
  Branches      61571    61769     +198     
============================================
+ Hits          84085    84398     +313     
- Misses       413674   414236     +562     
- Partials       9419     9436      +17

Flag	Coverage Δ
uitests	`3.93% <ø> (-0.04%)`	⬇️
unittests	`17.50% <56.76%> (+0.04%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

DaanHoogland · 2025-04-24T13:22:03Z

looks nice. would be great for end-users.

Adds new interface for image selection (template/iso) for an instance in UI. Old interface can still be used and it can be configured using UI configuration (config.json) OS categories/Guest OS categories have been improved with ability to create new categories, delete an existing category, and marking a category as featured to allow it to show up in the UI in the image selection interface. New APIs added: - addOsCategory - deleteOsCategory - updateOsCategory APIs updated: - updateOsType - listTemplates - listOsCategories Several improvements in UI especially related to forms - DeloyVM, ReinstallVM, CreateVnfAppliance, AddAutoscaleGroup. DeployVM form can now be opened from template details view with query params. Signed-off-by: Abhishek Kumar <[email protected]>

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr · 2025-04-28T13:28:58Z

@blueorangutan package

blueorangutan · 2025-04-28T13:30:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-04-28T15:02:55Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13202

show only required elements when query params are passed Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr · 2025-04-30T04:31:11Z

@blueorangutan package

blueorangutan · 2025-04-30T04:32:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-04-30T06:07:37Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13221

Signed-off-by: Abhishek Kumar <[email protected]>

Copilot

Pull Request Overview

This PR introduces an enhanced OS categorization interface for both the UI and API by adding new endpoints and updating existing ones to support operations such as adding, updating, and deleting OS categories. Key changes include:

New API commands for managing OS categories (add, update, delete).
Enhancements in the DAO and VO layers to support the new category properties such as featured and sortKey.
Updates to tests and response objects to incorporate the new fields.

Reviewed Changes

Copilot reviewed 55 out of 55 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
engine/schema/src/main/java/com/cloud/storage/dao/GuestOSDaoImpl.java	Added a new method to list IDs by category.
engine/schema/src/main/java/com/cloud/storage/GuestOSCategoryDaoImpl.java	Adjusted SQL count query for consistency.
engine/schema/src/main/java/com/cloud/storage/GuestOSCategoryVO.java	Included new columns (featured, sortKey) with their getters/setters.
api/.../*.java	Introduced and updated API commands and responses for managing OS categories.
api/src/main/java/org/apache/cloudstack/api/ApiConstants.java	Added new constant for ISO support.
api/src/main/java/com/cloud/server/ResourceTag.java	Added a new resource tag for Guest OS category.
api/src/main/java/org/apache/cloudstack/api/ResponseGenerator.java	Added response generation method for Guest OS categories.
api/src/main/java/org/apache/cloudstack/api/ManagementService.java	Declared new methods for OS category operations.

Copilot · 2025-04-30T11:35:22Z

api/src/main/java/org/apache/cloudstack/api/command/admin/guest/UpdateGuestOsCmd.java

@@ -71,7 +77,7 @@ public String getOsDisplayName() {
        return osDisplayName;
    }

-    public Map getDetails() {
+    public Map<String, String> getDetails() {


The getDetails() method initializes a new empty map and then checks if it is not empty, which will always return false, effectively ignoring the 'details' parameter intended to be processed. Also, remove the redundant semicolon in the map initialization ("new HashMap<>();;").

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr · 2025-04-30T14:27:19Z

@blueorangutan package

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr · 2025-06-04T13:00:10Z

@blueorangutan package

blueorangutan · 2025-06-04T13:02:03Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-04T14:11:13Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13602

shwstppr · 2025-06-04T16:18:46Z

@blueorangutan test

blueorangutan · 2025-06-04T16:20:03Z

@shwstppr a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

shwstppr · 2025-06-04T18:52:24Z

@blueorangutan test

blueorangutan · 2025-06-04T18:54:04Z

@shwstppr a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-06-05T08:16:18Z

[SF] Trillian test result (tid-13470)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54137 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10773-t13470-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2025-06-05T13:13:51Z

[SF] Trillian test result (tid-13471)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 63050 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10773-t13471-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

Signed-off-by: Abhishek Kumar <[email protected]>

kiranchavala

LGTM

Test Case Execution	Result
Test the add guest os category api and UI	Pass
Test the add guest os api and Ui	Pass
Ui : Test the vm deployment form	Pass
Test the ui improvements to the Templates/iso page	Pass
Test the guest os mapping	Pass

Screenshots

rohityadavcloud

LGTM - great UX feature. I've always found the template/ISO selection confusing. Hope this will improve that significantly.

Some statements to add new columns to cloud.guest_os_category have been added twice by mistake as part of #10773. These statements are idempotent and won't affect upgrade as such but better to cleanup Signed-off-by: Abhishek Kumar <[email protected]>

In PR #10773, 'AlmaLinux' was incorrectly written as 'Alma Linux' in the guest OS category name and OS classification. This PR corrects the naming to 'AlmaLinux'. Signed-off-by: Abhishek Kumar <[email protected]>

Adds new interface for image selection (template/iso) for an instance in UI. Old interface can still be used and it can be configured using UI configuration (config.json) OS categories/Guest OS categories have been improved with ability to create new categories, delete an existing category, and marking a category as featured to allow it to show up in the UI in the image selection interface. New APIs added: - addOsCategory - deleteOsCategory - updateOsCategory APIs updated: - updateOsType - listTemplates - listOsCategories Several improvements in UI especially related to forms - DeloyVM, ReinstallVM, CreateVnfAppliance, AddAutoscaleGroup. DeployVM form can now be opened from template/ISO details view with query params. Reorganized (removed and added some) OS categories to the following (in the same order): ``` 1. Ubuntu 2. Debian 3. Fedora 4. CentOS 5. Rocky Linux 6. Alma Linux 7. Oracle 8. RedHat 9. SUSE 10. Windows 11. Other ``` Documentation PR: apache/cloudstack-documentation#500 Signed-off-by: Abhishek Kumar <[email protected]>

Some statements to add new columns to cloud.guest_os_category have been added twice by mistake as part of apache#10773. These statements are idempotent and won't affect upgrade as such but better to cleanup Signed-off-by: Abhishek Kumar <[email protected]>

In PR apache#10773, 'AlmaLinux' was incorrectly written as 'Alma Linux' in the guest OS category name and OS classification. This PR corrects the naming to 'AlmaLinux'. Signed-off-by: Abhishek Kumar <[email protected]>

boring-cyborg bot added component:api component:management-server component:marvin component:storage component:UI Python Warning... Python code Ahead! labels Apr 24, 2025

shwstppr changed the title ~~Deployvm improvements~~ ui,api,server: template categorization based on os Apr 24, 2025

shwstppr mentioned this pull request Apr 28, 2025

notes on guest os category changes and modern image selection apache/cloudstack-documentation#500

Merged

shwstppr force-pushed the deployvm-improvements branch from f9852f0 to 90294a6 Compare April 28, 2025 12:35

reorganize os categories

7fc3865

Signed-off-by: Abhishek Kumar <[email protected]>

fixes for isoid as queryparam

6d2c124

show only required elements when query params are passed Signed-off-by: Abhishek Kumar <[email protected]>

tests

eda7679

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr requested a review from vishesh92 April 30, 2025 08:54

shwstppr added 2 commits April 30, 2025 15:08

allow updating category for multiple os

f2e1b57

Signed-off-by: Abhishek Kumar <[email protected]>

fix user template listing

14cf02a

Signed-off-by: Abhishek Kumar <[email protected]>

vishesh92 requested a review from Copilot April 30, 2025 11:34

Copilot AI reviewed Apr 30, 2025

View reviewed changes

shwstppr added 3 commits April 30, 2025 17:08

cosistent

3ad581e

Signed-off-by: Abhishek Kumar <[email protected]>

ui fixes

bcc64cc

Signed-off-by: Abhishek Kumar <[email protected]>

filter catedories for vnf templates

138c704

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr added 3 commits June 4, 2025 17:49

tests

a9c00ca

Signed-off-by: Abhishek Kumar <[email protected]>

tests

5e56234

Signed-off-by: Abhishek Kumar <[email protected]>

fix

ed5239f

Signed-off-by: Abhishek Kumar <[email protected]>

sureshanaparti added this to Apache CloudStack 4.21.0 Jun 5, 2025

fix lint eof

c5c2f37

Signed-off-by: Abhishek Kumar <[email protected]>

kiranchavala approved these changes Jun 10, 2025

View reviewed changes

rohityadavcloud marked this pull request as ready for review June 10, 2025 09:51

rohityadavcloud approved these changes Jun 10, 2025

View reviewed changes

rohityadavcloud closed this Jun 10, 2025

github-project-automation bot moved this to Done in Apache CloudStack 4.21.0 Jun 10, 2025

rohityadavcloud reopened this Jun 10, 2025

rohityadavcloud merged commit bce17b6 into apache:main Jun 10, 2025
43 of 49 checks passed

rohityadavcloud deleted the deployvm-improvements branch June 10, 2025 09:55

rohityadavcloud removed this from ACS 4.20.1 Jun 10, 2025

shwstppr mentioned this pull request Jun 11, 2025

engine-schema: fix duplicate statements in upgrade path #11001

Merged

14 tasks

shwstppr mentioned this pull request Jun 12, 2025

engine-schema: fix naming for AlmaLinux #11011

Merged

14 tasks

ui,api,server: template categorization based on os #10773

ui,api,server: template categorization based on os #10773

Uh oh!

Conversation

shwstppr commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecov bot commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

DaanHoogland commented Apr 24, 2025

Uh oh!

shwstppr commented Apr 28, 2025

Uh oh!

blueorangutan commented Apr 28, 2025

Uh oh!

blueorangutan commented Apr 28, 2025

Uh oh!

shwstppr commented Apr 30, 2025

Uh oh!

blueorangutan commented Apr 30, 2025

Uh oh!

blueorangutan commented Apr 30, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

shwstppr commented Apr 30, 2025

Uh oh!

shwstppr commented Jun 4, 2025

Uh oh!

blueorangutan commented Jun 4, 2025

Uh oh!

blueorangutan commented Jun 4, 2025

Uh oh!

shwstppr commented Jun 4, 2025

Uh oh!

blueorangutan commented Jun 4, 2025

Uh oh!

shwstppr commented Jun 4, 2025

Uh oh!

blueorangutan commented Jun 4, 2025

Uh oh!

blueorangutan commented Jun 5, 2025

Uh oh!

blueorangutan commented Jun 5, 2025

Uh oh!

kiranchavala left a comment

Choose a reason for hiding this comment

Uh oh!

rohityadavcloud left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

shwstppr commented Apr 24, 2025 •

edited

Loading

codecov bot commented Apr 24, 2025 •

edited

Loading