ds-identify not finding NoCloud, causes generator to disable service

[IsSkyfalls]

Bug report

cloud-init should run on boot, but is disabled by the generator.

Steps to reproduce the problem

this happens after every boot, including first boot.

Environment details

• Cloud-init version: /usr/bin/cloud-init 25.1.2
• Operating System Distribution: Archlinux
• Cloud provider, platform or installer type: VM on Proxmox, barebones archlinux install, with systemd-boot

logs

this is right after booting and logging in. I modified the generator to make it log the time. you can see that when I rerun ds-identify it worked fine. it seems to me that the generator is running too early when /dev/sr0 still haven't been loaded? sounds also kinda weird to me but it is the only explaination I have after quite a long time of debugging.
workaround: manually set /etc/cloud/ds-identify.cfg. the actual cloud-init service runs late enough when sr0 is available.

[root@pve-arch ~]# cat /run/cloud-init/ds-identify.log 
[up 1.87s] ds-identify 
policy loaded: mode=search report=false found=all maybe=all notfound=disabled
WARN: no datasource_list found, using default: MAAS ConfigDrive NoCloud AltCloud Azure Bigstep CloudSigma CloudStack DigitalOcean Vultr AliYun Ec2 GCE OpenNebula OpenStack VMware OVF SmartOS Scaleway Hetzner IBMCloud Oracle Exoscale RbxCloud UpCloud LXD NWCS Akamai WSL CloudCIX
DMI_PRODUCT_NAME=Standard PC (i440FX + PIIX, 1996)
DMI_SYS_VENDOR=QEMU
DMI_PRODUCT_SERIAL=
DMI_PRODUCT_UUID=dbe6a96f-fba5-44fc-828d-f0957ab14651
PID_1_PRODUCT_NAME=unavailable
DMI_CHASSIS_ASSET_TAG=
DMI_BOARD_NAME=unavailable
FS_LABELS=root,BOOT,BOOT
ISO9660_DEVS=
KERNEL_CMDLINE=initrd=initramfs-linux-lts.img root="LABEL=root" console=ttyS0,115200 console=tty0
VIRT=kvm
UNAME_KERNEL_NAME=Linux
UNAME_KERNEL_VERSION=#1 SMP PREEMPT_DYNAMIC Thu, 10 Jul 2025 15:17:11 +0000
UNAME_MACHINE=x86_64
DSNAME=
DSLIST=MAAS ConfigDrive NoCloud AltCloud Azure Bigstep CloudSigma CloudStack DigitalOcean Vultr AliYun Ec2 GCE OpenNebula OpenStack VMware OVF SmartOS Scaleway Hetzner IBMCloud Oracle Exoscale RbxCloud UpCloud LXD NWCS Akamai WSL CloudCIX
MODE=search
ON_FOUND=all
ON_MAYBE=all
ON_NOTFOUND=disabled
pid=198 ppid=180
is_container=false
is_ds_enabled(IBMCloud) = true.
ec2 platform is 'Unknown'.
is_ds_enabled(IBMCloud) = true.
No ds found [mode=search, notfound=disabled]. Disabled cloud-init [1]
[up 1.92s] returning 1

[root@pve-arch ~]# cat /run/cloud-init/cloud-init-generator.log 
/usr/lib/systemd/system-generators/cloud-init-generator normal=/run/systemd/generator early=/run/systemd/generator.early late=/run/systemd/generator.late
checking for datasource
Sat Jul 12 11:14:11 PM UTC 2025
ds-identify rc=1
cloud-init is enabled but no datasource found, disabling
already disabled: no change needed [no /run/systemd/generator.early/multi-user.target.wants/cloud-init.target]

[root@pve-arch ~]# journalctl -b | grep sr0
Jul 12 23:14:14 pve-arch kernel: sr 0:0:0:0: [sr0] scsi3-mmc drive: 16x/50x cd/rw xa/form2 cdda tray
Jul 12 23:14:14 pve-arch kernel: sr 0:0:0:0: Attached scsi CD-ROM sr0

[root@pve-arch ~]# blkid
/dev/sr0: BLOCK_SIZE="2048" UUID="2025-07-12-21-53-56-00" LABEL="cidata" TYPE="iso9660"
/dev/sda2: LABEL="root" UUID="cdd3dcd9-727f-4191-a282-ec4590ee1cdb" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="root" PARTUUID="44cd91cb-0d05-45d4-9cc5-5b14b1d97779"
/dev/sda1: LABEL_FATBOOT="BOOT" LABEL="BOOT" UUID="4FDD-93C5" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="boot" PARTUUID="3a49608a-c031-44ca-a9e3-b14ea5cbb052"

[root@pve-arch ~]# /usr/lib/cloud-init/ds-identify --force
WARN: no datasource_list found, using default: MAAS ConfigDrive NoCloud AltCloud Azure Bigstep CloudSigma CloudStack DigitalOcean Vultr AliYun Ec2 GCE OpenNebula OpenStack VMware OVF SmartOS Scaleway Hetzner IBMCloud Oracle Exoscale RbxCloud UpCloud LXD NWCS Akamai WSL CloudCIX

[root@pve-arch ~]# /usr/lib/systemd/system-generators/cloud-init-generator
[root@pve-arch ~]# cat /run/cloud-init/cloud-init-generator.log 
/usr/lib/systemd/system-generators/cloud-init-generator normal= early= late=
checking for datasource
Sat Jul 12 11:16:47 PM UTC 2025
ds-identify rc=0
enabled via /multi-user.target.wants/cloud-init.target -> /lib/systemd/system/cloud-init.target
removing /run/cloud-init/disabled and creating /run/cloud-init/enabled

[blackboxsw]

Thanks for those timestamps and for filing a bug to make cloud-init better. On first glance the symptoms appear to agree with your analysis. Your first boot ds-identify.log shows no ISO9660 devices: FS_LABELS=root,BOOT,BOOT ISO9660_DEVS= Your addition of the timestamp in ds-identify logging appears to confirm a 12 hour delay before /dev/sr0 is mounted. I'd have to agree that systemd generator timeframe isn't going to block for that amount of time awaiting a potential device mount labelled 'cidata'. Something else is going on with when QEMU is presenting that attached device that isn't going to help cloud-init get across the line in datasource detection, and no other QEMU artifacts such as kernel command line providing `ci.ds=nocloud` or dmi product serial set to `ds=nocloud`. I'm uncertain how we can help here at the moment. Do you have a set of reproducer steps that seem to work before cloud-init version 25.1.2 but appear broken on 25.1.2? Thank you for any further context.

…

On Sat, Jul 12, 2025 at 5:39 PM IsSkyfalls ***@***.***> wrote: *IsSkyfalls* created an issue (canonical/cloud-init#6304) <#6304> Bug report cloud-init should run on boot, but is disabled by the generator. Steps to reproduce the problem this happens after every boot, including first boot. Environment details - Cloud-init version: /usr/bin/cloud-init 25.1.2 - Operating System Distribution: Archlinux - Cloud provider, platform or installer type: VM on Proxmox, barebones archlinux install, with systemd-boot logs this is right after booting and logging in. I modified the generator to make it log the time. you can see that when I rerun ds-identify it worked fine. it seems to me that the generator is running too early when /dev/sr0 still haven't been loaded? sounds also kinda weird to me but it is the only explaination I have after quite a long time of debugging. workaround: manually set /etc/cloud/ds-identify.cfg. the actual cloud-init service runs late enough when sr0 is available. ***@***.*** ~]# cat /run/cloud-init/ds-identify.log [up 1.87s] ds-identify policy loaded: mode=search report=false found=all maybe=all notfound=disabled WARN: no datasource_list found, using default: MAAS ConfigDrive NoCloud AltCloud Azure Bigstep CloudSigma CloudStack DigitalOcean Vultr AliYun Ec2 GCE OpenNebula OpenStack VMware OVF SmartOS Scaleway Hetzner IBMCloud Oracle Exoscale RbxCloud UpCloud LXD NWCS Akamai WSL CloudCIX DMI_PRODUCT_NAME=Standard PC (i440FX + PIIX, 1996) DMI_SYS_VENDOR=QEMU DMI_PRODUCT_SERIAL= DMI_PRODUCT_UUID=dbe6a96f-fba5-44fc-828d-f0957ab14651 PID_1_PRODUCT_NAME=unavailable DMI_CHASSIS_ASSET_TAG= DMI_BOARD_NAME=unavailable FS_LABELS=root,BOOT,BOOT ISO9660_DEVS= KERNEL_CMDLINE=initrd=initramfs-linux-lts.img root="LABEL=root" console=ttyS0,115200 console=tty0 VIRT=kvm UNAME_KERNEL_NAME=Linux UNAME_KERNEL_VERSION=#1 SMP PREEMPT_DYNAMIC Thu, 10 Jul 2025 15:17:11 +0000 UNAME_MACHINE=x86_64 DSNAME= DSLIST=MAAS ConfigDrive NoCloud AltCloud Azure Bigstep CloudSigma CloudStack DigitalOcean Vultr AliYun Ec2 GCE OpenNebula OpenStack VMware OVF SmartOS Scaleway Hetzner IBMCloud Oracle Exoscale RbxCloud UpCloud LXD NWCS Akamai WSL CloudCIX MODE=search ON_FOUND=all ON_MAYBE=all ON_NOTFOUND=disabled pid=198 ppid=180 is_container=false is_ds_enabled(IBMCloud) = true. ec2 platform is 'Unknown'. is_ds_enabled(IBMCloud) = true. No ds found [mode=search, notfound=disabled]. Disabled cloud-init [1] [up 1.92s] returning 1 ***@***.*** ~]# cat /run/cloud-init/cloud-init-generator.log /usr/lib/systemd/system-generators/cloud-init-generator normal=/run/systemd/generator early=/run/systemd/generator.early late=/run/systemd/generator.late checking for datasource Sat Jul 12 11:14:11 PM UTC 2025 ds-identify rc=1 cloud-init is enabled but no datasource found, disabling already disabled: no change needed [no /run/systemd/generator.early/multi-user.target.wants/cloud-init.target] ***@***.*** ~]# journalctl -b | grep sr0 Jul 12 23:14:14 pve-arch kernel: sr 0:0:0:0: [sr0] scsi3-mmc drive: 16x/50x cd/rw xa/form2 cdda tray Jul 12 23:14:14 pve-arch kernel: sr 0:0:0:0: Attached scsi CD-ROM sr0 ***@***.*** ~]# blkid /dev/sr0: BLOCK_SIZE="2048" UUID="2025-07-12-21-53-56-00" LABEL="cidata" TYPE="iso9660" /dev/sda2: LABEL="root" UUID="cdd3dcd9-727f-4191-a282-ec4590ee1cdb" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="root" PARTUUID="44cd91cb-0d05-45d4-9cc5-5b14b1d97779" /dev/sda1: LABEL_FATBOOT="BOOT" LABEL="BOOT" UUID="4FDD-93C5" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="boot" PARTUUID="3a49608a-c031-44ca-a9e3-b14ea5cbb052" ***@***.*** ~]# /usr/lib/cloud-init/ds-identify --force WARN: no datasource_list found, using default: MAAS ConfigDrive NoCloud AltCloud Azure Bigstep CloudSigma CloudStack DigitalOcean Vultr AliYun Ec2 GCE OpenNebula OpenStack VMware OVF SmartOS Scaleway Hetzner IBMCloud Oracle Exoscale RbxCloud UpCloud LXD NWCS Akamai WSL CloudCIX ***@***.*** ~]# /usr/lib/systemd/system-generators/cloud-init-generator ***@***.*** ~]# cat /run/cloud-init/cloud-init-generator.log /usr/lib/systemd/system-generators/cloud-init-generator normal= early= late= checking for datasource Sat Jul 12 11:16:47 PM UTC 2025 ds-identify rc=0 enabled via /multi-user.target.wants/cloud-init.target -> /lib/systemd/system/cloud-init.target removing /run/cloud-init/disabled and creating /run/cloud-init/enabled — Reply to this email directly, view it on GitHub <#6304>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADLZJRVXREAF6O76WAV45CT3IGMDTAVCNFSM6AAAAACBMI4FD2VHI2DSMVQWIX3LMV43ASLTON2WKOZTGIZDKOJTGEYTGOI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>