Does this only work on freshly created disks?

[sickcodes]

Works fine on freshly created APFS, having trouble getting an existing Big Sur disk to mount read write however.

I can mount Big Sur using apfs-fuse read-only, can I help or any tips on forcing apfs-fuse to mount read/write using your module?

Or since you mentioned

o Encryption.
 o Compression, though the compressed contents of a file can be read from the
   'com.apple.decmpfs' and 'com.apple.ResourceFork' xattrs as long as they are
   under 64k.
 o Restoring to a snapshot.
 o Access control lists. This is not a priority.

Any tips on exact dimensions or specific block sizes that would be 100% editable with apfs-apfs-rw?

No Encryption is fine, does disk case sensitivity make a difference?

Or is the actual goal:

• raw image creation from a read-only disk copied over rsync -av or similar to make a disk that OSX will boot that you have manufactured?

I will try that today regardless^ also working on documentation

[eafer]

Hi. Case sensitivity shouldn't be a problem and any blocksize should be fine as long as it's smaller than a Linux page. That said, the module has only been tested with very simple containers, so I'm not surprised that you are running into problems. What's the issue exactly? Can you mount the disk read-only with my module? Are you getting any output on dmesg? Can you try running my fsck on your disk? That may tell us if it has an unsupported feature.

Right now I'm busy trying to implement a couple of features that we need urgently, so it may take me a while to start fixing bugs reported by users. I definitely want the module to be fully functional, but we are not quite there yet.

[sickcodes]

Hey @eafer! Thanks for replying so fast!

I'll do some tests on a fresh VPS with KVM so I can pinpoint the exact issue.

The device I was testing does have a bad superblock because I truncated it in Linux after shrinking it in OS X.

However I can still boot off it and mount using apfs-fuse read-only.

I cant mount using linux-apfs.

I'm going to test apfs-fuse in read only and rsync to a fresh disk and see if it will straight up just boot (KVM OpenCore macOS).

I'll also test a non-truncated disk obviously 😀

Will report back soon!

[sickcodes]

fdisk -l apfs.raw

# root@docker-ubuntu-c-16-nyc3-01:~# fdisk -l apfs.raw
# GPT PMBR size mismatch (104857599 != 39472135) will be corrected by write.
# Disk apfs.raw: 18.84 GiB, 20209733632 bytes, 39472136 sectors
# Units: sectors of 1 * 512 = 512 bytes
# Sector size (logical/physical): 512 bytes / 512 bytes
# I/O size (minimum/optimal): 512 bytes / 512 bytes
# Disklabel type: dos
# Disk identifier: 0x00000000

# Device     Boot Start      End  Sectors  Size Id Type
# apfs.raw1           1 39472135 39472135 18.8G ee GPT

git clone https://github.com/linux-apfs/apfsprogs.git
cd apfsprogs/apfsck
make
make install
cd ../mkapfs
make
make install
cd ../../
export PATH="~/bin:${PATH}"

git clone https://github.com/linux-apfs/linux-apfs-rw.git

cd linux-apfs-rw

make
make install

modprobe libcrc32c
insmod apfs.ko
cd ..

mkdir -p ./apfs-mnt


fsck.apfs apfs.raw
# Container superblock: block size is too small.

export DEVICE_MODEL=iMacPro1,1
export SERIAL=XXXXXXXXXXXX
export BOARD_SERIAL=XXXXXXXXXXXXXXXXX
export UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
export MAC_ADDRESS=00:03:93:00:00:00

export DISK_TWO="${PWD}/apfs.raw"
chmod 777 "$DISK_TWO"
docker run -it \
    --device /dev/kvm \
    -e "DISPLAY=${DISPLAY:-:0.0}" \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e RAM=10 \
    -p 50923:10022 \
    -e GENERATE_SPECIFIC=true \
    -e DEVICE_MODEL="${DEVICE_MODEL}" \
    -e SERIAL="${SERIAL}" \
    -e BOARD_SERIAL="${BOARD_SERIAL}" \
    -e UUID="${UUID}" \
    -e MAC_ADDRESS="${MAC_ADDRESS}" \
    -v "${IMAGE_NAME}":/image \
    -e WIDTH=1600 \
    -e HEIGHT=900 \
    -e MASTER_PLIST_URL='https://raw.githubusercontent.com/sickcodes/osx-serial-generator/master/config-custom.plist' \
    -v "${DISK_TWO}:/disktwo" \
    -e EXTRA='-device ide-hd,bus=sata.5,drive=DISKTWO -drive id=DISKTWO,if=none,file=/disktwo,format=raw' \
    sickcodes/docker-osx:big-sur

repaired all disks inside disk utility

Inside macOS

diskutil list
diskutil unmountDisk /dev/disk4
fsck_apfs -oy /dev/disk4
fsck_apfs -q /dev/disk4

# install apfs-fuse
apt install fuse libfuse3-dev bzip2 libbz2-dev cmake gcc cpp git libattr1-dev zlib1g-dev -y

git clone https://github.com/sgan81/apfs-fuse.git
cd apfs-fuse
git submodule init
git submodule update
mkdir build
cd build
cmake ..
make
make install

# root@docker-ubuntu-c-16-nyc3-01:~# apfsutil ./apfs.raw
# Found partitions:
# C12A7328-F81F-11D2-4BBA-00A0C93EC93B 78BA58BF-9638-4A49-45AE-E7CCE88F2420 0000000000000028 0000000000064027 0000000000000000 EFI System Partition
# 7C3457EF-0000-11AA-11AA-00306543ECAC DA201F5F-1AB2-4C33-1888-3C20F89518D6 0000000000064028 00000000025A4C07 0000000000000000 
# First APFS partition is 1

# Volume 0 BA38ED4B-947E-49C5-82AD-68B1E3A1343D
# ---------------------------------------------
# Role:               Data
# Name:               Docker-OSX-Raw-Big-Sur - Data (Case-sensitive)
# Capacity Consumed:  1903083520 Bytes
# FileVault:          No
# Snapshots:

# Volume 1 E5120E70-309F-4D55-B8E4-F914229FE4EA
# ---------------------------------------------
# Role:               Preboot
# Name:               Preboot (Case-insensitive)
# Capacity Consumed:  293552128 Bytes
# FileVault:          No
# Snapshots:

# Volume 2 476925AC-48F2-4ABB-9E0D-1069DAD79282
# ---------------------------------------------
# Role:               Recovery
# Name:               Recovery (Case-insensitive)
# Capacity Consumed:  613834752 Bytes
# FileVault:          No
# Snapshots:

# Volume 3 A12A3D2B-AA0E-41BD-9F31-C3F4219DE6A7
# ---------------------------------------------
# Role:               VM
# Name:               VM (Case-insensitive)
# Capacity Consumed:  1069056 Bytes
# FileVault:          No
# Snapshots:

# Volume 4 215F1FA9-D084-4F4C-BD36-B265E83F80F0
# ---------------------------------------------
# Role:               System
# Name:               Docker-OSX-Raw-Big-Sur (Case-insensitive)
# Capacity Consumed:  15050797056 Bytes
# FileVault:          No
# Snapshots:
#     640 : 'com.apple.os.update-35C699ADD439C653A3B09EBD8A30FBD366881244EB7C82207A2E135362FCCB7F'

# Volume 5 41683538-A64E-4C49-ABBD-211E0195838D
# ---------------------------------------------
# Role:               Update
# Name:               Update (Case-insensitive)
# Capacity Consumed:  520192 Bytes
# FileVault:          No
# Snapshots:




apfs-fuse apfs.raw ./apfs-mnt

# apfs.raw        19531248 17494892   2036356  90% /root/apfs-mnt


```bash
fdisk -l apfs.raw

GPT PMBR size mismatch (104857599 != 39472135) will be corrected by write.
Disk apfs.raw: 18.84 GiB, 20209733632 bytes, 39472136 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device     Boot Start      End  Sectors  Size Id Type
apfs.raw1           1 39472135 39472135 18.8G ee GPT

dmesg:

[14645.909610] squashfs: Unknown parameter 'vol'
[14645.912367] fuseblk: Unknown parameter 'vol'
[14645.914226] APFS (loop6): bad blocksize 0
[14648.022779] SGI XFS with ACLs, security attributes, realtime, no debug enabled
[14648.033440] JFS: nTxBlock = 8192, nTxLock = 65536
[14648.047356] ntfs: driver 2.1.32 [Flags: R/O MODULE].
[14648.064895] QNX4 filesystem 0.2.3 registered.

mount -o cknodes,vol=2 apfs.raw ./apfs-mnt

[14749.141759] squashfs: Unknown parameter 'cknodes'
[14749.143324] fuseblk: Unknown parameter 'cknodes'
[14749.144935] APFS (loop6): bad blocksize 0
[14749.146121] XFS (loop6): unknown mount option [cknodes].
[14749.146214] jfs: Unrecognized mount option "cknodes" or missing value

mount apfs.raw ./apfs-mnt

[14779.760602] APFS (loop6): bad blocksize 0

[sickcodes]

[user@hostname ~]$ fdisk -l /mac_hdd_ng_nonqcow_postreboot.img
Disk /mac_hdd_ng_nonqcow_postreboot.img: 50 GiB, 53687091200 bytes, 104857600 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5C21B10B-9C3E-4AE9-8482-F4C321A8094D

Device                                                              Start       End   Sectors  Size Type
/mac_hdd_ng_nonqcow_postreboot.img1     40    409639    409600  200M EFI System
/mac_hdd_ng_nonqcow_postreboot.img2 409640 104857559 104447920 49.8G Apple APFS

mkdir /tmp/mnting

sudo mount /mac_hdd_ng_nonqcow_postreboot.img /tmp/mnting
# mount: /tmp/mnting: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.

[19855.349011] loop0: detected capacity change from 0 to 104857600

mkdir /tmp/mnting

sudo mount /mac_hdd_ng_nonqcow_postreboot.img /tmp/mnting ; sudo dmesg  | tail -n100

[19855.349011] loop0: detected capacity change from 0 to 104857600

sudo mount -t apfs mac_hdd_ng_nonqcow_postreboot.img /tmp/mnting

[19994.269280] loop0: detected capacity change from 0 to 104857600
[19994.335625] APFS (loop0): bad blocksize 0

sudo mount -t apfs -o cknodes mac_hdd_ng_nonqcow_postreboot.img /tmp/mnting

[20040.352851] loop0: detected capacity change from 0 to 104857600
[20040.399571] APFS (loop0): bad blocksize 0

sudo mount -t apfs -o cknodes,vol=1 mac_hdd_ng_nonqcow_postreboot.img /tmp/mnting

[20112.751350] loop0: detected capacity change from 0 to 104857600
[20112.803452] APFS (loop0): bad blocksize 0

sudo mount -t apfs -o cknodes,vol=2 mac_hdd_ng_nonqcow_postreboot.img /tmp/mnting

[20139.823348] loop0: detected capacity change from 0 to 104857600
[20139.851010] APFS (loop0): bad blocksize 0