Improve `qvm-backup-restore --paranoid-mode` #358

alimirjamali · 2025-05-15T09:20:25Z

Before openning a Shell window in the DispVM, check for availability of core-admin-client in the DisposableVM and show error message if not available

resolves: QubesOS/qubes-issues#9962

ben-grande · 2025-05-15T10:08:18Z

Use command -v (POSIX) instead of which, which might not be present by default on Alpine Linux.

alimirjamali · 2025-05-15T10:15:45Z

Use command -v (POSIX) instead of which, which might not be present by default on Alpine Linux.

Justed tested it on Alpine. which is indeed a part of Busybox. But OK. I will switch the code to command -v as soon as the current iteration of CI/CD tasks is complete.

ben-grande · 2025-05-15T10:18:41Z

Justed tested it on Alpine. which is indeed a part of Busybox.

Oh, ok, they might have their own implementation because when I saw a separate package, the Required by did not give me much clues, but that package is for the Gnu version of which? While busybox which is probably builtin then.

codecov · 2025-05-15T10:20:52Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.72%. Comparing base (c89ed14) to head (302533f).
Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #358      +/-   ##
==========================================
+ Coverage   75.68%   75.72%   +0.03%     
==========================================
  Files          52       52              
  Lines        9028     9033       +5     
==========================================
+ Hits         6833     6840       +7     
+ Misses       2195     2193       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

alimirjamali · 2025-05-15T10:28:37Z

Justed tested it on Alpine. which is indeed a part of Busybox.

Oh, ok, they might have their own implementation because when I saw a separate package, the Required by did not give me much clues, but that package is for the Gnu version of which? While busybox which is probably builtin then.

which of GNU Coreutils, which of Busybox and which of MIT licensed Rust rewrite of uutils (Ubuntu is switching to it). Finally I have a good reason to to break all of English language grammar and say: Which which which works \o/

alimirjamali · 2025-05-15T12:30:49Z

PipelineRetryFailed

Minimalist73 · 2025-05-16T08:55:04Z

I'm currently on Qubes 4.2 and the -q option of which doesn't exist in dom0:

$ which -q qvm-backup-restore
/usr/bin/which: invalid option -- 'q'
/usr/bin/qvm-backup-restore

Is it different on 4.3? I tried inside a Fedora 41 app qube but it's also missing from there so I'm not sure.

Before openning a Shell window in the DispVM, check for availability of `core-admin-client` in the DisposableVM and show error message if not available resolves: QubesOS/qubes-issues#9962

alimirjamali · 2025-05-16T09:49:09Z

I'm currently on Qubes 4.2 and the -q option of which doesn't exist in dom0:
$ which -q qvm-backup-restore
/usr/bin/which: invalid option -- 'q'
/usr/bin/qvm-backup-restore
Is it different on 4.3? I tried inside a Fedora 41 app qube but it's also missing from there so I'm not sure.

Yes. You are absolutely right. The -q is not a which option at all. I must have mixed it up with grep -q.

Anyways, I switched the check from which to command -v as was suggested by @ben-grande . One less call to external binaries. And command -v works in both bash and zsh (which is the default in Whonix templates)

qubesos-bot · 2025-05-19T04:56:55Z

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025051904-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025031804-4.3&flavor=update

system_tests_whonix
- whonix_torbrowser: unnamed test (unknown)
- whonix_torbrowser: Failed (test died)
  # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...
system_tests_kde_gui_interactive
- gui_keyboard_layout: wait_serial (wait serial expected)
  # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...
- gui_keyboard_layout: Failed (test died)
  # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...
system_tests_audio@hw1
- TC_20_AudioVM_PipeWire_debian-12-xfce: test_250_audio_playback_audiovm_pipewire (failure)
  AssertionError: Command 'timeout 20s pw-play --raw --format=f32 --r...
system_tests_gui_tools@hw7
- qui_widgets_devices: unnamed test (unknown)
- qui_widgets_devices: Failed (test died)
  # Test died: no candidate needle with tag(s) 'qui-devices-opened' m...
system_tests_qwt_win10_seamless@hw13
- windows_clipboard_and_filecopy: unnamed test (unknown)
- windows_clipboard_and_filecopy: Failed (test died)
  # Test died: no candidate needle with tag(s) 'windows-Edge-address-...
system_tests_qwt_win11@hw13
- windows_install: wait_serial (wait serial expected)
  # wait_serial expected: qr/dcWzE-\d+-/...
- windows_install: Failed (test died + timed out)
  # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

Failed tests

11 failures

system_tests_whonix
- whonix_torbrowser: unnamed test (unknown)
- whonix_torbrowser: Failed (test died)
  # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...
system_tests_kde_gui_interactive
- gui_keyboard_layout: wait_serial (wait serial expected)
  # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...
- gui_keyboard_layout: Failed (test died)
  # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...
system_tests_audio@hw1
- TC_20_AudioVM_PipeWire_debian-12-xfce: test_250_audio_playback_audiovm_pipewire (failure)
  AssertionError: Command 'timeout 20s pw-play --raw --format=f32 --r...
system_tests_gui_tools@hw7
- qui_widgets_devices: unnamed test (unknown)
- qui_widgets_devices: Failed (test died)
  # Test died: no candidate needle with tag(s) 'qui-devices-opened' m...
system_tests_qwt_win10_seamless@hw13
- windows_clipboard_and_filecopy: unnamed test (unknown)
- windows_clipboard_and_filecopy: Failed (test died)
  # Test died: no candidate needle with tag(s) 'windows-Edge-address-...
system_tests_qwt_win11@hw13
- windows_install: wait_serial (wait serial expected)
  # wait_serial expected: qr/dcWzE-\d+-/...
- windows_install: Failed (test died + timed out)
  # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/132953#dependencies

14 fixed

system_tests_whonix
- whonixcheck: fail (unknown)
  Whonixcheck for sys-whonix failed...
- whonixcheck: unnamed test (unknown)
system_tests_suspend
- suspend: unnamed test (unknown)
- suspend: Failed (test died)
  # Test died: no candidate needle with tag(s) 'SUSPEND-FAILED' match...
system_tests_basic_vm_qrexec_gui
- TC_20_NonAudio_whonix-gateway-17: test_300_bug_1028_gui_memory_pinning (failure)
  AssertionError: Dom0 window doesn't match VM window content, saved ...
system_tests_qrexec
- TC_00_Qrexec_fedora-41-xfce: test_081_qrexec_service_argument_allow_specific (error)
  subprocess.CalledProcessError: Command '/usr/lib/qubes/qrexec-clien...
system_tests_kde_gui_interactive
- clipboard_and_web: unnamed test (unknown)
- clipboard_and_web: Failed (test died)
  # Test died: no candidate needle with tag(s) 'qubes-website' matche...
- clipboard_and_web: wait_serial (wait serial expected)
  # wait_serial expected: "lspci; echo 2E8vz-\$?-"...
system_tests_guivm_vnc_gui_interactive
- gui_filecopy: unnamed test (unknown)
- gui_filecopy: Failed (test died)
  # Test died: no candidate needle with tag(s) 'files-work' matched...
system_tests_audio
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_252_audio_playback_audiovm_switch_hvm (failure)
  AssertionError: only silence detected, no useful audio data
system_tests_whonix@hw7
- whonixcheck: fail (unknown)
  Whonixcheck for sys-whonix failed...
- whonixcheck: unnamed test (unknown)

Unstable tests

system_tests_update
update2/Failed (1/5 times with errors)
- job 139051 # Test died: command 'script -c 'qubes-vm-update --force-update --l...
system_tests_update@hw1
update2/Failed (1/5 times with errors)
- job 139051 # Test died: command 'script -c 'qubes-vm-update --force-update --l...
system_tests_update@hw7
update2/Failed (1/5 times with errors)
- job 139051 # Test died: command 'script -c 'qubes-vm-update --force-update --l...
system_tests_update@hw13
update2/Failed (1/5 times with errors)
- job 139051 # Test died: command 'script -c 'qubes-vm-update --force-update --l...

Performance Tests

Performance degradation:

13 performance degradations

dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 183156.00 :small_red_triangle: ( previous job: 446963.00, degradation: 40.98%)
dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 81844.00 :small_red_triangle: ( previous job: 129298.00, degradation: 63.30%)
dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 233120.00 :small_red_triangle: ( previous job: 294295.00, degradation: 79.21%)
dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 71505.00 :small_red_triangle: ( previous job: 79803.00, degradation: 89.60%)
dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 3947.00 :small_red_triangle: ( previous job: 6149.00, degradation: 64.19%)
dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 4252.00 :small_red_triangle: ( previous job: 4903.00, degradation: 86.72%)
fedora-41-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 3046.00 :small_red_triangle: ( previous job: 3599.00, degradation: 84.63%)
fedora-41-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 53948.00 :small_red_triangle: ( previous job: 61534.00, degradation: 87.67%)
fedora-41-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 1690.00 :small_red_triangle: ( previous job: 2215.00, degradation: 76.30%)
fedora-41-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 101259.00 :small_red_triangle: ( previous job: 179949.00, degradation: 56.27%)
fedora-41-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 282102.00 :small_red_triangle: ( previous job: 324737.00, degradation: 86.87%)
fedora-41-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 3462.00 :small_red_triangle: ( previous job: 5672.00, degradation: 61.04%)
fedora-41-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 6917.00 :small_red_triangle: ( previous job: 7867.00, degradation: 87.92%)

Remaining performance tests:

43 tests

debian-12-xfce_exec: 7.14 🔺 ( previous job: 7.12, degradation: 100.35%)
debian-12-xfce_exec-root: 26.89 🟢 ( previous job: 28.65, improvement: 93.83%)
debian-12-xfce_socket: 8.09 🟢 ( previous job: 8.60, improvement: 94.00%)
debian-12-xfce_socket-root: 9.14 🔺 ( previous job: 8.52, degradation: 107.21%)
debian-12-xfce_exec-data-simplex: 66.19 🟢 ( previous job: 71.62, improvement: 92.41%)
debian-12-xfce_exec-data-duplex: 71.15 🔺 ( previous job: 70.34, degradation: 101.15%)
debian-12-xfce_exec-data-duplex-root: 68.97 🟢 ( previous job: 82.72, improvement: 83.38%)
debian-12-xfce_socket-data-duplex: 149.64 🟢 ( previous job: 156.96, improvement: 95.34%)
fedora-41-xfce_exec: 9.19 🟢 ( previous job: 9.27, improvement: 99.18%)
fedora-41-xfce_exec-root: 61.29 🟢 ( previous job: 61.51, improvement: 99.64%)
fedora-41-xfce_socket: 8.55 🟢 ( previous job: 8.63, improvement: 99.12%)
fedora-41-xfce_socket-root: 8.83 🔺 ( previous job: 8.71, degradation: 101.41%)
fedora-41-xfce_exec-data-simplex: 73.14 🟢 ( previous job: 75.53, improvement: 96.83%)
fedora-41-xfce_exec-data-duplex: 71.81 🔺 ( previous job: 71.56, degradation: 100.36%)
fedora-41-xfce_exec-data-duplex-root: 103.58 🟢 ( previous job: 109.13, improvement: 94.92%)
fedora-41-xfce_socket-data-duplex: 140.94 🟢 ( previous job: 150.61, improvement: 93.58%)
dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 113777.00 :green_circle: ( previous job: 95454.00, improvement: 119.20%)
dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 10353.00 :small_red_triangle: ( previous job: 10795.00, degradation: 95.91%)
dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 4659.00 :small_red_triangle: ( previous job: 4826.00, degradation: 96.54%)
dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 499559.00 :green_circle: ( previous job: 382273.00, improvement: 130.68%)
dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 240499.00 :small_red_triangle: ( previous job: 250795.00, degradation: 95.89%)
dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 434013.00 :small_red_triangle: ( previous job: 437636.00, degradation: 99.17%)
dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 194707.00 :green_circle: ( previous job: 184752.00, improvement: 105.39%)
dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 101566.00 :green_circle: ( previous job: 62195.00, improvement: 163.30%)
dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 6182.00 :small_red_triangle: ( previous job: 6479.00, degradation: 95.42%)
dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 8044.00 :green_circle: ( previous job: 7669.00, improvement: 104.89%)
fedora-41-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 386785.00 :green_circle: ( previous job: 368309.00, improvement: 105.02%)
fedora-41-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 147477.00 :small_red_triangle: ( previous job: 162081.00, degradation: 90.99%)
fedora-41-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 310321.00 :small_red_triangle: ( previous job: 318716.00, degradation: 97.37%)
fedora-41-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 93369.00 :green_circle: ( previous job: 87940.00, improvement: 106.17%)
fedora-41-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 91325.00 :green_circle: ( previous job: 82694.00, improvement: 110.44%)
fedora-41-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 7740.00 :small_red_triangle: ( previous job: 8485.00, degradation: 91.22%)
fedora-41-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 944.00 :green_circle: ( previous job: 542.00, improvement: 174.17%)
fedora-41-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 379918.00 :green_circle: ( previous job: 373957.00, improvement: 101.59%)
fedora-41-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 225888.00 :green_circle: ( previous job: 170062.00, improvement: 132.83%)
fedora-41-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 310046.00 :small_red_triangle: ( previous job: 334687.00, degradation: 92.64%)
fedora-41-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 89225.00 :green_circle: ( previous job: 80283.00, improvement: 111.14%)
fedora-41-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 9010.00 :green_circle: ( previous job: 7540.00, improvement: 119.50%)
fedora-41-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 1288.00 :green_circle: ( previous job: 1130.00, improvement: 113.98%)
fedora-41-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 385364.00 :green_circle: ( previous job: 369868.00, improvement: 104.19%)
fedora-41-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 43557.00 :green_circle: ( previous job: 17567.00, improvement: 247.95%)
fedora-41-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 79244.00 :green_circle: ( previous job: 79021.00, improvement: 100.28%)
fedora-41-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 2336.00 :green_circle: ( previous job: 1953.00, improvement: 119.61%)

alimirjamali force-pushed the issue-9962-restore-paranoid-mode branch from 4e08825 to c96c995 Compare May 15, 2025 09:49

alimirjamali force-pushed the issue-9962-restore-paranoid-mode branch 2 times, most recently from d3f62ef to c2c3cc2 Compare May 15, 2025 11:49

Improve qvm-backup-restore --paranoid-mode

302533f

Before openning a Shell window in the DispVM, check for availability of `core-admin-client` in the DisposableVM and show error message if not available resolves: QubesOS/qubes-issues#9962

alimirjamali force-pushed the issue-9962-restore-paranoid-mode branch from c2c3cc2 to 302533f Compare May 16, 2025 09:17

marmarek added the openqa-pending label May 18, 2025

qubesos-bot mentioned this pull request May 19, 2025

Fix handling remove event QubesOS/qubes-app-linux-input-proxy#38

Merged

This was referenced May 19, 2025

Fix tests for changes in mock API QubesOS/qubes-manager#423

Merged

Allow VM boot modes to change a VM's default user QubesOS/qubes-core-admin#681

Merged

marmarek approved these changes May 19, 2025

View reviewed changes

marmarek merged commit 16834f9 into QubesOS:main May 20, 2025
4 of 5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Improve `qvm-backup-restore --paranoid-mode` #358

Improve `qvm-backup-restore --paranoid-mode` #358

Uh oh!

alimirjamali commented May 15, 2025

Uh oh!

ben-grande commented May 15, 2025

Uh oh!

alimirjamali commented May 15, 2025

Uh oh!

ben-grande commented May 15, 2025

Uh oh!

codecov bot commented May 15, 2025 •

edited

Loading

Uh oh!

alimirjamali commented May 15, 2025

Uh oh!

alimirjamali commented May 15, 2025

Uh oh!

Minimalist73 commented May 16, 2025

Uh oh!

alimirjamali commented May 16, 2025

Uh oh!

qubesos-bot commented May 19, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Improve qvm-backup-restore --paranoid-mode #358

Improve qvm-backup-restore --paranoid-mode #358

Uh oh!

Conversation

alimirjamali commented May 15, 2025

Uh oh!

ben-grande commented May 15, 2025

Uh oh!

alimirjamali commented May 15, 2025

Uh oh!

ben-grande commented May 15, 2025

Uh oh!

codecov bot commented May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

alimirjamali commented May 15, 2025

Uh oh!

alimirjamali commented May 15, 2025

Uh oh!

Minimalist73 commented May 16, 2025

Uh oh!

alimirjamali commented May 16, 2025

Uh oh!

qubesos-bot commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

OpenQA test summary

New failures, excluding unstable

Failed tests

Fixed failures

Unstable tests

Performance Tests

Performance degradation:

Remaining performance tests:

Uh oh!

Uh oh!

Uh oh!

Improve `qvm-backup-restore --paranoid-mode` #358

Improve `qvm-backup-restore --paranoid-mode` #358

codecov bot commented May 15, 2025 •

edited

Loading

qubesos-bot commented May 19, 2025 •

edited

Loading