Clonezilla and Chrome OS Flex/Chromium

Hello All,

I work on Chrome OS Flex https://chromeenterprise.google/os/chromeosflex/,
a version of Google's Chrome OS that is designed to be installable on any
x86 device. We've had a number of requests from customers and partners to
produce a Clonezilla disk image of Flex, for easy mass deployment of the OS
to large groups of machines. I've been trying to successfully create and
restore an image using Clonezilla Live 2.8.1-12, but am having issues due
to the unusually complex partition layout of Chrome OS
https://chromium.googlesource.com/chromiumos/docs/+/HEAD/disk_format.md,
which Clonezilla seems to be having trouble interpreting.

I am able to successfully capture a disk image by using the -scpt "skip
checking the partition table" flag, but no combination of expert-mode flags
that I can find results in a successful restore of the image back to disk,
even on the same machine with the same internal storage that the image was
captured from. Every combination results in a variation of an "unknown
partition table format" error. I have verified that the storage device is
not corrupted, and the OS itself on the source storage works fine.

Has anyone had experience with getting Clonezilla to operate more akin to
dd - aka, not attempt to read and understand a disk at the partition level,
and just create a "dumb" bit-perfect image of the data on the disk? Are
there any other tools that could create a "dumb" image that Clonezilla can
make use of?

Any help would be much appreciated

If you want a non-standard partition layout (for no obvious reason), you will have to use non-standard tools. Good luck with "the root FS shall not exceed 1G".

Hello @steven_shiau, below is the behavior that we observe when trying to use Clonezilla to capture and restore a Chrome OS Flex image in both beginner and advanced mode:

Beginner mode capture:

Image capture: fails with “the partition table on this disk is illegal/invalid” and "error: Can’t have overlapping partitions”

Expert mode capture:

Command: ocs-sr -q1 -c -j2 -scpt -z1p -i 0 -sfsck -scs -senc
Error: same as with beginner mode, but continuing past the partition table errors will allow an image file to be captured successfully.

Expert mode restore:

Command: ocs-sr -g auto -e1 auto -e2 -c -r -j2 -scr -p choose restoredisk attempt1 sda

Error: Unknown partition table format from file /home/partimag/attempt1/sda-pt.parted! “ocs-live-general” finished with error!

Next steps: Could definitely use some guidance on what combination of expert mode commands might allow a successful restore here. It seems like Clonezilla keeps getting tripped up by the unusual partition scheme of Chrome OS, which includes a number of zero-size partitions and other unusual features, Is there a way to force it to use dd for both capture and restore, so that it does not attempt to understand the partition structure and just reads and writes the raw data?

Have you tried to use "-j0" or --create-part-by-dd?
j0, --create-part-by-dd Use dd to dump the partition table from saved image instead of sfdisk. ///Note/// This does NOT work when logical drives exist.

Steven

Hello @steven_shiau

Unfortunately the image capture process fails with the same error using both the --create-part-by-dd and -j0 arguments. The error is "can't have overlapping partitions" by parted, and "unknown partition table format" error from the overall process.

We've discovered something interesting - an older version of our OS can be captured and restored just fine by Clonezilla, in beginner mode with default settings. The newer version, with a tweaked partition layout, is what is failing repeatedly. Is there anything about the 2 partition layouts attached, that jumps out as potentially causing the issue?

Not sure. Clonezilla will parse the file sda-pt.parted in the image dir. Hence please attach this file here:
/home/partimag/attemp1/sda-pt.parted
so that I can download and test it here.

Steven

Thanks Steven, please see attached sda-pt.parted from an unsuccessful image capture of Chrome OS Flex, installed to the disk depicted in the screenshots in my previous posts.

Since there isn't much to see in sda-pt.parted, attached are all of the files that result from the failure.

That's weird... So when you run:
sudo parted -s /dev/sda print
it shows all the correct info about the partitions.
However, the file sda-pt.parted from your image is just:
sda-pt.parted
Model: ATA TS32GSSD370S (scsi)
Disk /dev/sda: 62533296s
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

This is very weird...
Actually the sda-pt.parted is dumped by this command:
sudo parted -s /dev/sda unit s print > sda-pt.parted
So maybe you can try to rebuild that in your image dir.
//NOTE// Backup your image before you modify it. Just in case.

Steven

Hello @steven_shiau,

Very interesting, so it seems like the problem lies with parted. Attached is the result when I manually run the requested commands. It seems like the errors in the overall process are being generated by parted.

If I launch parted manually from the command line, type "print all" and then manually press "I" to ignore the resulting "can't have overlapping partitions" error, I see what looks like normal output with the details of partitions on the disk.

Apparently when using script (-s) in parted, there is no any option we can assign to ignore the error/warning message. Therefore I suggest you manually create a file sda-pt.parted in the image dir, and its output is what you have got from the interactive mode of parted.
Remember to issue "unit s" before you run "print" command.
A complete sda-pt.parted is like:

Model: VMware, VMware Virtual S (scsi)
Disk /dev/sda: 41943040s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
1 2048s 1050623s 1048576s fat32 EFI System Partition boot, esp
2 1050624s 41940991s 40890368s ext4

The "Partition Table: gpt" is very important in your case, which Clonezilla parses to get the format.

As for "sda-pt.parted.compact", use "unit compact" before you run "print" in parted.
Happy hacking!

Steven

Hello everyone,

I'm wondering if I can get a confirmation on the command that should work for generating the sda-pt.parted file to be generated separately, then placed with the OS Flex image backup. I have tried with the following combinations while in a clonezilla shell

sudo parted /dev/sda unit s print > sda-pt.parted
sudo parted -s /dev/sda unit s print > sda-pt.parted

The output I get from these commands is the same "Error: Can't have overlapping partitions.". Is there an extra switch required?

In Parted's changelog I found that they added an option for skipping this error around 2014, but can't find any documentation around it. The changelog says

"You can now choose to ignore errors about partitions that overlap,
or are longer than the disk. This allows you to use parted to
repair the problem."

Any help would be appreciated,
James

Ok @steven_shiau, I followed your advice and did the following:

1. Ran an unsuccessful beginner mode capture, which failed as expected
2. Manually ran parted /dev/sda unit s print and parted /dev/sda unit compat print in interactive mode, chose to ignore the error, and captured the results. Attached for reference.
3. Overwrote the sda-pt.parted and sda-pt.parted.compact files in the folder for the failed capture
4. Re-ran the capture command from the command line, pointing to the same directory

Unfortunately, step 4 fails, and it appears that parted overwrites my new, correct files with the empty files. Is there a way to have the process skip running parted, so that it parses my corrected sda-pt.parted and sda-pt.parted.compact files?

What did you mean by "Re-ran the capture command from the command line, pointing to the same directory" in step 4?
Did you mean you save the image using Clonezilla ocs-sr command again?
If so, no, you should not do that. What you need to is to overwrite an existing image with the steps 2&3. I.e., to hack the image dir. Do not use ocs-sr command to save the image again.
Once you have the hacked image, you can use ocs-sr command to "restore" the hacked image.

Steven

Hello @steven_shiau

Definitely don't entirely understand your guidance here.  

Yes, I am referring to running the ocs-sr command again in step 4 to try and get the image save process to successfully complete.

There is no previously successful image to "hack" - the ocs-sr capture command fails before any actual partition data is captured.  See attached screenshot for what is created before the image capture/save process fails. What we are trying to do is find a way for the image to be successfully saved, we haven't even made it to the restore step yet. 

OK, my bad. Since I do not have such a device here so that I can not try it before I gave you the hacking tips.
In that case, I suggest you try to edit the file /usr/share/drbl/sbin/ocs-functions before you capture the image. In the function "save_part_table_mbr_dm_info", around the lines 8788 and 8790, remove the option "-s". i.e., make them like:

    LC_ALL=C parted /dev/$ihd unit s print > $target_dir_fullpath/$(to_filename ${ihd})-pt.parted

and

    LC_ALL=C parted /dev/$ihd unit compact print > $target_dir_fullpath/$(to_filename ${ihd})-pt.parted.compact

Then when you capture the image, when parted is run, it should be in interactive mode. You will have to press "I" to ignore that. This might successfully create the required files.
I did not test this, but I think it should work.

Steven

Hello,

I tried this today. Unless I'm mistaken, the files that need editing are in the filesystem.squashfs file in /live/filesystem.squashfs folder. These are the steps I did.

1. Download the latest clonezilla stable - 2.8.1-12 as a .iso
2. Write the ISO to a USB stick using rufus
3. Transfer the /live/filesystem.squashfs file to a linux machine
4. On linux, install squashfs-tools sudo apt-get install squashfs-tools
5. Transfer filesystem.squashfs to the linux machines /var/tmp directory ( WinSCP )
6. Run the command sudo unsquashfs filesystem.squashfs
7. sudo nano /var/tmp/squashfs-root/usr/share/drbl/sbin/ocs-functions
8. Hit ctrl + shift + _ to go to line number. Enter 8875
9. Remove the -s after parted on lines 8875 and 8877
10. Rebuild filesystem.squashfs from /var/tmp/ - sudo mksquashfs squashfs-root/ filesystem.squashfs -noappend -always-use-fragments
11. Transfer the rebuilt filesystem.squashfs back over to the clonezilla USB's live folder
12. Live boot to clonezilla and use beginner mode to capture.

It stills shows the same error when it starts ot make the image and doesn't have interactive mode enabled for parted. Is there something I'm missing?

James

If you can provide a machine that I can access that with ssh login (You can make it after booting Clonezilla live, configure the internet and start sshd service), please email me at
steven at clonezilla org
With that, it's easier for us to reproduce this issue and try to find a solution for that.

Steven

The issue seems to be that the version of parted included with clonezilla doesn't have an option to manually skip errors, or at least doesn't skip errors by default. I've tried this with clonezilla shell as well. Due to that, the advice to remove -s from the parted command doesn't change the result.

Without getting into setting up machines and SSH, is there a way that you can confirm whether Clonezilla's parted has a skip errors manually option?

James

I might have a workaround for this. However, I need the environment to test it. If you have a virtual machine with Chrome OS Flex installed, please share the virtual disk so that I can try to implement this workaround and test it.

Steven

Hi Steven,

Thank you. I'm not from the OS Flex team, just independently looking into the same issue, so they may have a better example machine or VM for you to work from.

James