KVM: Option to deploy a VM with existing volume/snapshot #10503

slavkap · 2025-03-05T11:39:18Z

Description

This PR allows the users to deploy virtual machines with an existing volume or a snapshot.
For now it is limited only for KVM hypervisors and Zone-wide storage pools

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale

Major
Minor

How Has This Been Tested?

Environment
OS - Rocky Linux 8.9
2 Zones with one agent host on each
hypervisor KVM

Tested manually and with smoke tests with StorPool and NFS as a primary storage

codecov · 2025-03-05T11:46:42Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 3.90%. Comparing base (16c60c7) to head (b36e05b).

❗ There is a different number of reports uploaded between BASE (16c60c7) and HEAD (b36e05b). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (16c60c7) HEAD (b36e05b)

unittests 1 0

Additional details and impacted files

@@              Coverage Diff              @@
##               main   #10503       +/-   ##
=============================================
- Coverage     16.57%    3.90%   -12.68%     
=============================================
  Files          5743      414     -5329     
  Lines        510494    33746   -476748     
  Branches      62075     6075    -56000     
=============================================
- Hits          84617     1317    -83300     
+ Misses       416415    32273   -384142     
+ Partials       9462      156     -9306

Flag	Coverage Δ
uitests	`3.90% <ø> (ø)`
unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

DaanHoogland · 2025-03-06T10:43:13Z

@blueorangutan package

blueorangutan · 2025-03-06T10:44:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-03-06T11:51:10Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12681

DaanHoogland · 2025-03-06T12:21:04Z

@blueorangutan test

blueorangutan · 2025-03-06T12:22:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-03-07T06:41:11Z

[SF] Trillian test result (tid-12598)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 63251 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t12598-kvm-ol8.zip
Smoke tests completed. 140 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup	`Error`	1520.36	test_network.py
test_01_deploy_vm_with_existing_volume	`Error`	7.34	test_vm_lifecycle_with_snapshot_or_volume.py
test_02_deploy_vm_with_existing_snapshot	`Error`	0.06	test_vm_lifecycle_with_snapshot_or_volume.py
test_03_deploy_vm_with_existing_volume_deleted_template	`Failure`	861.05	test_vm_lifecycle_with_snapshot_or_volume.py
test_04_deploy_vm_with_existing_snapshot_deleted_template	`Failure`	862.76	test_vm_lifecycle_with_snapshot_or_volume.py
test_05_deploy_vm_with_existing_snapshot_deleted_volume	`Failure`	847.20	test_vm_lifecycle_with_snapshot_or_volume.py

DaanHoogland · 2025-03-10T14:58:53Z

@blueorangutan package

blueorangutan · 2025-03-10T15:00:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-03-10T16:08:38Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12727

DaanHoogland · 2025-03-11T07:27:42Z

@blueorangutan test

blueorangutan · 2025-03-11T07:30:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-03-12T01:27:01Z

[SF] Trillian test result (tid-12645)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 61921 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t12645-kvm-ol8.zip
Smoke tests completed. 140 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup	`Error`	1520.50	test_network.py
test_01_deploy_vm_with_existing_volume	`Error`	0.00	test_vm_lifecycle_with_snapshot_or_volume.py
test_02_deploy_vm_with_existing_snapshot	`Error`	0.00	test_vm_lifecycle_with_snapshot_or_volume.py
test_03_deploy_vm_with_existing_volume_deleted_template	`Error`	0.00	test_vm_lifecycle_with_snapshot_or_volume.py
test_04_deploy_vm_with_existing_snapshot_deleted_template	`Error`	0.00	test_vm_lifecycle_with_snapshot_or_volume.py
test_05_deploy_vm_with_existing_snapshot_deleted_volume	`Error`	0.00	test_vm_lifecycle_with_snapshot_or_volume.py

DaanHoogland · 2025-03-12T10:01:15Z

@slavkap the tests fail because the self doesn't have a snapshot that was supposed to be created in the class setup method. Is this a timing issue?

slavkap · 2025-03-12T11:18:54Z

@blueorangutan package

blueorangutan · 2025-03-12T11:20:04Z

@slavkap a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-03-12T12:24:59Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12756

github-actions · 2025-06-19T07:27:57Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

rajujith

LGTM
Tested new instance deployment from a detached ROOT Volume and a volume snapshot.

check if the hypervisor is KVM check if the primary storage's scope is ZONE wide

…t KVM

add StorPool tags to a volume created from snapshot or to a volume which will be attached as a ROOT to a new VM

Add the StorPool's tags when volume is created from a snapshot or a volume is attached as a ROOT to a VM Addressed reviews

Added new line

DaanHoogland · 2025-06-24T18:47:59Z

@blueorangutan package

blueorangutan · 2025-06-24T18:50:04Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-24T19:59:30Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13901

test/integration/smoke/test_vm_lifecycle_with_snapshot_or_volume.py

test/integration/plugins/storpool/test_storpool_tiers.py

DaanHoogland · 2025-06-25T07:28:43Z

@blueorangutan test

blueorangutan · 2025-06-25T07:30:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-06-25T23:25:39Z

[SF] Trillian test result (tid-13598)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54578 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t13598-kvm-ol8.zip
Smoke tests completed. 141 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_deployVMInSharedNetwork	`Failure`	472.97	test_network.py

slavkap · 2025-06-27T07:17:17Z

Hi all,
I believe that the test failure is not related to the changes in the PR. It fails while trying to list the mount path with the error:

AssertionError: False is not true : After umount directory should be empty but contains: ['sudo: unable to resolve host VM-5cc1cc35-742a-46bc-ae8c-6785a06829bf: Temporary failure in name resolution']

nvazquez

Code LGTM - haven't tested it. Only one minor non-blocking comment left

api/src/main/java/org/apache/cloudstack/api/command/user/vm/DeployVMCmd.java

DaanHoogland · 2025-06-27T14:54:55Z

@blueorangutan package

blueorangutan · 2025-06-27T14:56:06Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-27T16:50:47Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13933

DaanHoogland · 2025-06-30T06:31:06Z

@blueorangutan test

blueorangutan · 2025-06-30T06:32:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-06-30T23:12:55Z

[SF] Trillian test result (tid-13629)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 57115 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t13629-kvm-ol8.zip
Smoke tests completed. 142 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

boring-cyborg bot added component:api component:compute component:integration-test component:kubernetes component:marvin component:networking component:orchestration component:secondary-storage component:ssvm component:storage component:vpc Python Warning... Python code Ahead! labels Mar 5, 2025

bernardodemarco added this to the 4.21.0 milestone Mar 5, 2025

rajujith approved these changes Jun 24, 2025

View reviewed changes

slavkap added 6 commits June 24, 2025 17:37

Option to deploy a VM with existing volume/snapshot

f25e0cb

smoke test changes

6a74187

check if the hypervisor is KVM check if the primary storage's scope is ZONE wide

skip all tests if the storage isn't Zone-Wide and the hypervisor isn'…

66bfcd3

…t KVM

support StorPool tags

599e24b

add StorPool tags to a volume created from snapshot or to a volume which will be attached as a ROOT to a new VM

Add StorPool tags on the new ROOT volume

c5f8de3

Add the StorPool's tags when volume is created from a snapshot or a volume is attached as a ROOT to a VM Addressed reviews

addressed review

b8d9dcb

Added new line

slavkap force-pushed the deploy-vm-with-existing-volume-or-snapshot branch from d8b4872 to b8d9dcb Compare June 24, 2025 14:43

github-actions bot removed the status:has-conflicts label Jun 24, 2025

DaanHoogland reviewed Jun 25, 2025

View reviewed changes

Addressed comments

a1f75ab

nvazquez approved these changes Jun 27, 2025

View reviewed changes

api/src/main/java/org/apache/cloudstack/api/command/user/vm/DeployVMCmd.java Outdated Show resolved Hide resolved

Addressed review

b36e05b

DaanHoogland mentioned this pull request Jul 4, 2025

Deploy VM from Snapshot #4492

Open

DaanHoogland linked an issue Jul 6, 2025 that may be closed by this pull request

Deploy VM from Snapshot #4492

Open

slavkap mentioned this pull request Jul 8, 2025

UI support for deploy a VM from volume/snapshot #11164

Open

9 tasks

KVM: Option to deploy a VM with existing volume/snapshot #10503

Are you sure you want to change the base?

KVM: Option to deploy a VM with existing volume/snapshot #10503

Uh oh!

Conversation

slavkap commented Mar 5, 2025

Description

Types of changes

Feature/Enhancement Scale

How Has This Been Tested?

Uh oh!

codecov bot commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

DaanHoogland commented Mar 6, 2025

Uh oh!

blueorangutan commented Mar 6, 2025

Uh oh!

blueorangutan commented Mar 6, 2025

Uh oh!

DaanHoogland commented Mar 6, 2025

Uh oh!

blueorangutan commented Mar 6, 2025

Uh oh!

blueorangutan commented Mar 7, 2025

Uh oh!

DaanHoogland commented Mar 10, 2025

Uh oh!

blueorangutan commented Mar 10, 2025

Uh oh!

blueorangutan commented Mar 10, 2025

Uh oh!

DaanHoogland commented Mar 11, 2025

Uh oh!

blueorangutan commented Mar 11, 2025

Uh oh!

blueorangutan commented Mar 12, 2025

Uh oh!

DaanHoogland commented Mar 12, 2025

Uh oh!

slavkap commented Mar 12, 2025

Uh oh!

blueorangutan commented Mar 12, 2025

Uh oh!

blueorangutan commented Mar 12, 2025

Uh oh!

github-actions bot commented Jun 19, 2025

Uh oh!

rajujith left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Jun 24, 2025

Uh oh!

blueorangutan commented Jun 24, 2025

Uh oh!

blueorangutan commented Jun 24, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DaanHoogland commented Jun 25, 2025

Uh oh!

blueorangutan commented Jun 25, 2025

Uh oh!

blueorangutan commented Jun 25, 2025

Uh oh!

slavkap commented Jun 27, 2025

Uh oh!

nvazquez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DaanHoogland commented Jun 27, 2025

Uh oh!

codecov bot commented Mar 5, 2025 •

edited

Loading