Automated macOS installation for VMs

by Jakub Jirůtka

Source: https://www.deviantart.com/kossnocorp/art/Hackintosh-logo-130022106

InstallFest 2024

“Holistic software engineer”

Works at FEE CTU in Prague

FOSS developer & contributor

Alpine Linux developer

$~ whoami

@jirutka

@jakub@jirutka.cz

@JakubJirutka

Source: https://lezebre.lu/en/sticker-simons-cat-claws/

This presentation is for educational purposes only. Techniques shown may not comply with Apple’s License Agreement. Use at your own risk. No liability assumed.

Disclaimer

preconfigured VM images for CI and also for developers

deterministic and fully documented environment

periodic and fully automatic regeneration

supporting multiple versions

Why?

Source: ChatGPT 4 (DALL-E 2)

building and testing applications for macOS (on CI)

But why macOS?

Source: ChatGPT 4 (DALL-E 2)

Source: ChatGPT 4 (DALL-E 2) trained on art stolen from the Internet

How does this work with Linux?

https://github.com/alpinelinux/alpine-make-vm-image

alpine-make-vm-image

Script to create custom Alpine Linux disk images for x86_64 and aarch64 virtual machines.

~400 LoC in POSIX shell

Uses just apk-tools, qemu-img, qemu-nbd, chroot, runs on any Linux.

> apk not found, downloading static apk-tools
URL:https://gitlab.alpinelinux.org/api/v4/projects/5/packages/generic/v2.14.0/x86_64/apk.static [4336800/4336800] -> "apk.static" [1]
apk.static: OK

> Creating qcow2 image of size 2G
Formatting 'alpine-bios-2024-01-06.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2147483648 lazy_refcounts=off refcount_bits=16

> Attaching image alpine-bios-2024-01-06.qcow2 as a NBD device

> Creating filesystem(s)
mke2fs 1.46.5 (30-Dec-2021)
64-bit filesystem support is not enabled.  The larger fields afforded by this feature enable full-strength checksumming.  Pass -O 64bit to rectify.
Creating filesystem with 524288 4k blocks and 131072 inodes
Filesystem UUID: 35cd1afa-4e8f-4331-aebc-bf6f72fa3661
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912

Allocating group tables:  0/16  done                            
Writing inode tables:  0/16  done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information:  0/16  done


> Mounting image at /tmp/alpine-make-vm-image.5zC7DM

> Installing base system
# ./apk.static add --root $rootdir --initdb --arch x86_64 alpine-base
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/latest-stable/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/latest-stable/community/x86_64/APKINDEX.tar.gz
(1/25) Installing alpine-baselayout-data (3.4.3-r2)
(2/25) Installing musl (1.2.4_git20230717-r4)
(3/25) Installing busybox (1.36.1-r15)
Executing busybox-1.36.1-r15.post-install
(4/25) Installing busybox-binsh (1.36.1-r15)
(5/25) Installing alpine-baselayout (3.4.3-r2)
Executing alpine-baselayout-3.4.3-r2.pre-install
Executing alpine-baselayout-3.4.3-r2.post-install
(6/25) Installing ifupdown-ng (0.12.1-r4)
(7/25) Installing libcap2 (2.69-r1)
(8/25) Installing openrc (0.52.1-r1)
Executing openrc-0.52.1-r1.post-install
(9/25) Installing mdev-conf (4.6-r0)
(10/25) Installing busybox-mdev-openrc (1.36.1-r15)
(11/25) Installing alpine-conf (3.17.0-r0)
(12/25) Installing alpine-keys (2.4-r1)
(13/25) Installing alpine-release (3.19.0-r0)
(14/25) Installing ca-certificates-bundle (20230506-r0)
(15/25) Installing libcrypto3 (3.1.4-r2)
(16/25) Installing libssl3 (3.1.4-r2)
(17/25) Installing ssl_client (1.36.1-r15)
(18/25) Installing zlib (1.3-r2)
(19/25) Installing apk-tools (2.14.0-r5)
(20/25) Installing busybox-openrc (1.36.1-r15)
(21/25) Installing busybox-suid (1.36.1-r15)
(22/25) Installing scanelf (1.3.7-r2)
(23/25) Installing musl-utils (1.2.4_git20230717-r4)
(24/25) Installing libc-utils (0.7.2-r5)
(25/25) Installing alpine-base (3.19.0-r0)
Executing busybox-1.36.1-r15.trigger
OK: 10 MiB in 25 packages
(1/5) Installing libblkid (2.39.3-r0)
(2/5) Installing libcom_err (1.47.0-r5)
(3/5) Installing e2fsprogs-libs (1.47.0-r5)
(4/5) Installing libuuid (2.39.3-r0)
(5/5) Installing e2fsprogs (1.47.0-r5)
Executing busybox-1.36.1-r15.trigger
OK: 11 MiB in 30 packages

> Installing kernel linux-virt
(1/12) Installing xz-libs (5.4.5-r0)
(2/12) Installing zstd-libs (1.5.5-r8)
(3/12) Installing kmod (31-r2)
(4/12) Installing kmod-openrc (31-r2)
(5/12) Installing lddtree (1.27-r0)
(6/12) Installing argon2-libs (20190702-r5)
(7/12) Installing device-mapper-libs (2.03.23-r0)
(8/12) Installing json-c (0.17-r0)
(9/12) Installing cryptsetup-libs (2.6.1-r8)
(10/12) Installing kmod-libs (31-r2)
(11/12) Installing mkinitfs (3.9.0-r0)
Executing mkinitfs-3.9.0-r0.post-install
(12/12) Installing linux-virt (6.6.9-r0)
Executing busybox-1.36.1-r15.trigger
Executing kmod-31-r2.trigger
Executing mkinitfs-3.9.0-r0.trigger
==> initramfs: creating /boot/initramfs-virt for 6.6.9-0-virt
OK: 58 MiB in 42 packages

> Installing and configuring mkinitfs

> Setting up extlinux bootloader
(1/3) Installing mtools (4.0.43-r1)
(2/3) Installing blkid (2.39.3-r0)
(3/3) Installing syslinux (6.04_pre1-r15)
OK: 62 MiB in 45 packages
/boot is device /dev/nbd15
Warning: unable to obtain device geometry (defaulting to 64 heads, 32 sectors)
         (on hard disks, this is usually harmless.)
/boot is device /dev/nbd15
Warning: unable to obtain device geometry (defaulting to 64 heads, 32 sectors)
         (on hard disks, this is usually harmless.)

> Configuring system

> Enabling base system services
 * service devfs added to runlevel sysinit
 * service dmesg added to runlevel sysinit
 * service mdev added to runlevel sysinit
 * service hwdrivers added to runlevel sysinit
 * service cgroups added to runlevel sysinit
 * service modules added to runlevel boot
 * service hwclock added to runlevel boot
 * service swap added to runlevel boot
 * service hostname added to runlevel boot
 * service sysctl added to runlevel boot
 * service bootmisc added to runlevel boot
 * service syslog added to runlevel boot
 * service seedrng added to runlevel boot
 * service killprocs added to runlevel shutdown
 * service savecache added to runlevel shutdown

 * service mount-ro added to runlevel shutdown
> Installing additional packages
(1/30) Installing gmp (6.3.0-r0)
(2/30) Installing nettle (3.9.1-r0)
(3/30) Installing libunistring (1.1-r2)
(4/30) Installing libidn2 (2.3.4-r4)
(5/30) Installing libffi (3.4.4-r3)
(6/30) Installing libtasn1 (4.19.0-r2)
(7/30) Installing p11-kit (0.25.3-r0)
(8/30) Installing gnutls (3.8.1-r0)
(9/30) Installing libseccomp (2.5.5-r0)
(10/30) Installing chrony (4.5-r0)
Executing chrony-4.5-r0.pre-install
(11/30) Installing chrony-openrc (4.5-r0)
(12/30) Installing doas (6.8.2-r6)
(13/30) Installing doas-sudo-shim (0.1.1-r1)
(14/30) Installing ncurses-terminfo-base (6.4_p20231125-r0)
(15/30) Installing libncursesw (6.4_p20231125-r0)
(16/30) Installing less (643-r1)
(17/30) Installing libacl (2.3.1-r4)
(18/30) Installing popt (1.19-r3)
(19/30) Installing logrotate (3.21.0-r1)
(20/30) Installing logrotate-openrc (3.21.0-r1)
(21/30) Installing openssh-keygen (9.6_p1-r0)
(22/30) Installing libedit (20230828.3.1-r3)
(23/30) Installing openssh-client-common (9.6_p1-r0)
(24/30) Installing openssh-client-default (9.6_p1-r0)
(25/30) Installing openssh-sftp-server (9.6_p1-r0)
(26/30) Installing openssh-server-common (9.6_p1-r0)
(27/30) Installing openssh-server-common-openrc (9.6_p1-r0)
(28/30) Installing openssh-server (9.6_p1-r0)
(29/30) Installing openssh (9.6_p1-r0)
(30/30) Installing ssmtp (2.64-r20)
Executing busybox-1.36.1-r15.trigger
OK: 76 MiB in 75 packages

> Copying content of /home/runner/work/alpine-make-vm-image/alpine-make-vm-image/example/rootfs into image
./
usr/
usr/local/
usr/local/bin/
usr/local/bin/hello

> Executing script in chroot: configure.sh 
Linux fv-az520-878 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 Linux

1) Set up timezone

2) Set up networking

3) Adjust rc.conf

4) Enable services
 * service acpid added to runlevel default
 * service chronyd added to runlevel default
 * service crond added to runlevel default
 * service net.eth0 added to runlevel default
 * service net.lo added to runlevel boot
 * service termencoding added to runlevel boot

5) List /usr/local/bin
total 12
drwxr-xr-x    2 root     root          4096 Jan  6 19:08 .
drwxr-xr-x    5 root     root          4096 Jan  6 19:08 ..
-rwxr-xr-x    1 root     root            32 Jan  6 19:08 hello

> Completed
-rw-r--r-- 1 root root 174M Jan  6 18:08 alpine-bios-2024-01-06.qcow2
/dev/nbd15 disconnected

11seconds

From 0 to customised bootable VM image with

So how about macOS…?

Source: ChatGPT 4 (DALL-E 2) trained on art stolen from the Internet

It’s a UNIX system…

Source: https://www.youtube.com/watch?v=JOeY07qKU9c

…or is it?

Source: ChatGPT 4 (DALL-E 2) trained on art stolen from the Internet

BSD

Source: https://en.wikipedia.org/wiki/Architecture_of_macOS

macOS architecture

XNU

QEMU ≥ 6.2.0
CPU with Intel VT-x / AMD SVM, SSE4.1 and AVX2
OpenCore – bootloader (successor of Clover)
- https://github.com/kholia/OSX-KVM

Problems:

virtio-blk is supported, but virtio-scsi is not
macOS is sensitive to TSC (must be monotonic)
running macOS on QEMU on QEMU doesn’t work

What’s needed to boot macOS on QEMU/KVM?

QEMU parameters

qemu-system-x86_64 \
  -enable-kvm -machine q35 -m 4096 -smp 4,cores=2,sockets=1 \
  -cpu Skylake-Server,kvm=on,vendor=GenuineIntel,vmware-cpuid-freq=on,\
    +invtsc,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check \
  -device isa-applesmc,\
    osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" \
  -usb -device usb-kbd -device usb-tablet \
  -device usb-ehci,id=ehci \
  -device nec-usb-xhci,id=xhci \
  -global nec-usb-xhci.msi=off \
  -drive if=pflash,format=raw,readonly=on,file=OVMF_CODE.fd \
  -drive if=pflash,format=raw,file=OVMF_VARS.fd \
  -smbios type=2 \
  -device ich9-ahci,id=sata \
  -drive id=OpenCoreBoot,if=none,snapshot=on,format=qcow2,file=OpenCore.qcow2 \
  -device ide-hd,bus=sata.2,drive=OpenCoreBoot \
  -drive id=InstallMedia,if=none,file=BaseSystem.img,format=raw \
  -device ide-hd,bus=sata.3,drive=InstallMedia \
  -drive id=MacHDD,if=none,file=MacHDD.qcow2,format=qcow2 \
  -device ide-hd,bus=sata.4,drive=MacHDD \
  -netdev user,id=net0,hostfwd=tcp::2222-:22 -device virtio-net-pci,netdev=net0,\
    id=net0,mac=52:54:00:c9:18:27 \
  -monitor stdio \
  -device vmware-svga

https://github.com/kholia/OSX-KVM/

our hard work by these words guarded please dont steal

(c)AppleComputerInc

Source: ChatGPT 4 (DALL-E 2)

CI infrastructure

IaaS

GitLab

Runner Manager

API

KVM

Hardware

Host Linux Kernel

Guest OS

QEMU

libvirtd

Guest OS

QEMU

Guest OS

QEMU

CI Pipeline

installer-image-fetch

opencore-image-fetch

check-images

setup

customize

finalize

install

CI Pipeline

installer-image-fetch

opencore-image-fetch

check-images

setup

customize

finalize

install

Where/how to get macOS installation?

Source: ChatGPT 4 (DALL-E 2) trained on art stolen from the Internet

Hidden partition in every Apple device running macOS.

Downloads installation data from osrecovery.apple.com.

macOS Internet Recovery

Script fetch-macOS-v2.py from github.com/kholia/OSX-KVM.

Can run on Linux.

BaseSystem.dmg – bootable image

macOS Internet Recovery

-bash-3.2# ./startosinstall ...
Preparing: 100%
Restarting...

Error: system reboot failed

Source: https://archive.org/details/cat-meme/03c9ee9c0a5f15908ea183bf0b4bea98.png

macOS Internet Recovery

osinstallersetupd[535]: OSISPredicateUpdateProduct: OS Installation Payload:
  Multiple matching products found; choosing the one with the latest post date

osinstallersetupd[535]: OSISPredicateUpdateProduct: <IAMSUProduct: key=042-95491,
  name="macOS Ventura", evaluated=YES, installable=YES, staged=NO>: Found product: OS Installation Payload

...

osinstallersetupd[535]: IA OS Version: 13.6.2
osinstallersetupd[535]: IA OS Build: 22G2321

...

osishelperd_intel[537]: -[OSISHelper blessUpdateBundleAtPath:mutableProductPath:enclosingMountPoint:withReply:]
  failed to copy /Volumes/System/macOS Install Data/UpdateBundle/AssetData/
  ./boot/Firmware/System/Library/CoreServices/bootbase.efi

osinstallersetupd[535]: -[OSISHelperProxy blessUpdateBundleAtPath:mutableProductPath:enclosingMountPoint:]_block_invoke_2: Error Domain=NSCocoaErrorDomain Code=260
  "The file “bootbase.efi” couldn’t be opened because there is no such file." UserInfo={NSFilePath=/Volumes/System/macOS Install Data/UpdateBundle/AssetData/./boot/Firmware/System/Library/CoreServices/bootbase.efi, NSUnderlyingError=0x6000017753b0 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}

macOS Internet Recovery

The build 22G2321 is for the MacBook Pro with M3 that shipped with macOS Ventura.

The problem right now is that the installer is listing the available Monterey versions, and picking the newest compatible one. The newest one is currently the M2-only update, which is causing the issue, but it shouldn’t be picking this as it is incompatible.

Source: https://www.reddit.com/r/homelab/comments/qg37an/comment/idx3eyk/

But this topic is not about Hackintosh, this is happening on genuine MacBook!

Download full (offline) macOS installation

Script gibMacOS.py from github.com/corpnewt/gibMacOS.

./gibMacOS.py --print-json --device-id VMM-x86_64 --version 14.1 --no-interactive
./gibMacOS.py --version 14.1 --build 23B74 --no-interactive

InstallAssistant.pkg – macOS package

Can be downloaded from AppStore – requires login.

InstallAssistant.pkg

$ du InstallAssistant.pkg
12.7G  InstallAssistant.pkg

$ file InstallAssistant.pkg
InstallAssistant.pkg: xar archive compressed TOC: 4313, SHA-1 checksum

$ 7z x -t xar InstallAssistant.pkg

$ ls
-rw-r--r-- 269.3K Bom
-rw-r--r--    918 PackageInfo
-rw-r--r--  11.1M Payload
-rw-r--r--    643 Scripts
-rw-r--r--  12.7G SharedSupport.dmg
-rw-r--r--   8.9K [TOC].xml

$ file SharedSupport.dmg                                                                                                                                                                                                                          git
SharedSupport.dmg: DOS/MBR boot sector, code offset 0x58+2, OEM-ID "BSD  4.4", sectors/track 32, heads 16, sectors 409600 (volumes > 32 MB), FAT (32 bit), sectors/FAT 3151, serial number 0x67e317ed, label: "EFI        "

$ dmg2img -l SharedSupport.dmg
SharedSupport.dmg --> (partition list)

partition 0: Protective Master Boot Record (MBR : 0)
partition 1: GPT Header (Primary GPT Header : 1)
partition 2: GPT Partition Data (Primary GPT Table : 2)
partition 3:  (Apple_Free : 3)
partition 4: EFI System Partition (C12A7328-F81F-11D2-BA4B-00A0C93EC93B : 4)
partition 5: disk image (Apple_HFS : 5)
partition 6:  (Apple_Free : 6)
partition 7: GPT Partition Data (Backup GPT Table : 7)
partition 8: GPT Header (Backup GPT Header : 8)

$ 7z x SharedSupport.dmg
7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20
 64-bit locale=C.UTF-8 Threads:12 OPEN_MAX:1024

Scanning the drive for archives:
1 file, 13605194978 bytes (13 GiB)

Extracting archive: SharedSupport.dmg
--
Path = SharedSupport.dmg
Type = Dmg
Physical Size = 13605194978
Method = Zero0 Copy Zero2 CRC
Blocks = 98
----
Path = 5.hfs
Size = 14005383168
Packed Size = 13605063168
Comment = disk image (Apple_HFS : 5)
Method = Zero0 Copy Zero2 CRC
--
Path = 5.hfs
Type = HFS
Physical Size = 14005383168
Method = HFS+
Cluster Size = 4096
Free Space = 365486080
Created = 2024-02-28 22:59:11
Modified = 2024-02-29 08:03:59

Everything is Ok

Folders: 5
Files: 83
Size:       13610941804
Compressed: 13605194978

$ cd Shared\ Support
$ tree -h
[ 376]  .
├── [ 181]  InstallInfo.plist
├── [   0]  [HFS+ Private Data]
├── [6.8K]  com_apple_MobileAsset_MacSoftwareUpdate
│   ├── [ 18K]  0de8db5c435029ba8569f7600d6545fdaaf1f010.json
│   ├── [ 18K]  0e897fa22b5abfee6159343e3d1df6671b77f6a9.json
│   ├── ...
│   ├── [ 13G]  a8d9e8271da55b7529281b7aadbaf9796b375556.zip
│   ├── [1.0M]  com_apple_MobileAsset_MacSoftwareUpdate.xml
│   └── ...
└── [ 300]  com_apple_MobileAsset_MobileSoftwareUpdate_MacUpdateBrain
    ├── [ 12K]  433ad65ffe546a9a89571bda141e7978838eac8a.json
    ├── [3.7M]  433ad65ffe546a9a89571bda141e7978838eac8a.zip
    └── [1.2K]  com_apple_MobileAsset_MobileSoftwareUpdate_MacUpdateBrain.xml

4 directories, 81 files

$ file a8d9e8271da55b7529281b7aadbaf9796b375556.zip
a8d9...56.zip: Zip archive data, at least v2.0 to extract, compression method=store

$ 7z x a8d9e8271da55b7529281b7aadbaf9796b375556.zip
$ tree -h --du
[ 13G]  .
├── [ 13G]  AssetData
│   ├── [2.1K]  Info.plist
│   ├── [7.9M]  Restore
│   │   ├── [7.5K]  022-15129-328.chunklist
│   │   ├── [ 328]  AppleDiagnostics.chunklist
│   │   ├── [2.7M]  AppleDiagnostics.dmg
│   │   ├── [4.4K]  BaseSystem.chunklist
│   │   └── [5.1M]  BootabilityBundle
│   │       └── ...
│   ├── [1.2G]  boot
│   │   ├── [  32]  BridgeVersion.bin
│   │   ├── [ 522]  BridgeVersion.plist
│   │   ├── [9.6M]  BuildManifest.plist
│   │   ├── [ 76M]  EFI
│   │   │   ├── [ 11M]  AMDFirmware
│   │   │   │   ├── [157K]  GpuUtil.efi
│   │   │   │   └── [ 11M]  Payloads
│   │   │   │       ├── [256K]  D05001A1XG.011
│   │   │   │       ├── ...
│   │   │   │       └── [512K]  V20DonguilA1XTEG2_047_186_M465.sb
│   │   │   ├── [265K]  AppleSDFirmware
│   │   │   │   ├── [112K]  7A09.ROM
│   │   │   │   ├── [148K]  SDFWUpdater.efi
│   │   │   │   ├── [1.2K]  UniversalSDFWUpdater.plist
│   │   │   │   └── [3.9K]  config.ini
│   │   │   ├── ...
│   │   │   ├── [3.7K]  SMCJSONs
│   │   │   │   ├── ...
│   │   │   │   └── [  62]  Mac-FFE5EF870D7BA81A.json
│   │   │   ├── [ 18M]  SMCPayloads
│   │   │   │   ├── ...
│   │   │   │   ├── [519K]  Mac-FFE5EF870D7BA81A
│   │   │   │   │   ├── [ 31K]  Mac-FFE5EF870D7BA81A.epm
│   │   │   │   │   ├── [357K]  Mac-FFE5EF870D7BA81A.smc
│   │   │   │   │   ├── [ 93K]  flasher_base.smc
│   │   │   │   │   └── [ 38K]  flasher_update.smc
│   │   │   │   └── [185K]  SmcFlasher.efi
│   │   │   └── [ 25M]  USBCUpdater
│   │   │       ├── [ 87K]  HPMUtil.efi
│   │   │       ├── ...
│   │   │       ├── [504K]  Mac-EE2EBD4B90B839A8
│   │   │       │   ├── [ 702]  Config.plist
│   │   │       │   └── [503K]  Mac-EE2EBD4B90B839A8-5.92.0.bin
│   │   │       └── [218K]  ThorUtil.efi
│   │   ├── [920M]  Firmware
│   │   │   ├── ...
│   │   │   ├── [ 56M]  096-17380-285.dmg.x86.mtree
│   │   │   ├── [ 229]  096-17380-285.dmg.x86.root_hash
│   │   │   ├── [336K]  096-17380-285.dmg.x86.trustcache
│   │   │   ├── [157K]  AMDFirmware
│   │   │   │   └── [157K]  GpuUtil.efi
│   │   │   ├── [ 11M]  AOP
│   │   │   │   ├── ...
│   │   │   │   └── [1.9M]  aopfw-mac15saop.im4p
│   │   │   │ ...
│   │   │   └── [ 15K]  x86_64SURamDisk.dmg.x86.trustcache
│   │   ├── [1.6K]  PlatformSupport.plist
│   │   ├── [ 21K]  Restore.plist
│   │   ├── [ 360]  RestoreVersion.plist
│   │   ├── [ 60M]  System
│   │   │   └── [ 60M]  Library
│   │   │       └── [ 60M]  KernelCollections
│   │   │           └── [ 60M]  BootKernelExtensions.kc
│   │   ├── [ 600]  SystemVersion.plist
│   │   ├── [ 26M]  kernelcache.release.mac13g
│   │   ├── ...
│   │   ├── [ 18M]  kernelcache.release.vma2
│   │   └── [ 14K]  usr
│   │       └── [ 14K]  standalone
│   │           └── [ 14K]  bootcaches.plist
│   ├── [  14]  payload
│   │   └── [   0]  replace
│   ├── [437K]  payload.bom
│   ├── [ 128]  payload.bom.signature
│   ├── [ 12G]  payloadv2
│   │   ├── [1.2G]  basesystem_patches
│   │   │   ├── [537M]  arm64eBaseSystem.dmg
│   │   │   ├── [4.2M]  arm64eBaseSystem.dmg.ecc
│   │   │   ├── [674M]  x86_64BaseSystem.dmg
│   │   │   └── [2.4M]  x86_64BaseSystem.dmg.ecc
│   │   ├── [  12]  data_payload
│   │   ├── [1.9K]  firmlinks_payload
│   │   ├── [2.7M]  fixup.manifest
│   │   ├── [4.1G]  image_patches
│   │   │   ├── [9.0M]  cryptex-app
│   │   │   ├── [3.0G]  cryptex-system-arm64e
│   │   │   └── [1.1G]  cryptex-system-x86_64
│   │   ├── [ 16M]  links.txt
│   │   ├── [734K]  payload.000
│   │   ├── [1.3M]  payload.000.ecc
│   │   ├── ...
│   │   ├── [9.7M]  payload.042
│   │   ├── [2.1M]  payload.042.ecc
│   │   ├── [ 592]  payload_chunks.txt
│   │   └── [  12]  prepare_payload
│   ├── [472K]  payloadv2.bom
│   ├── [ 128]  payloadv2.bom.signature
│   ├── [ 43M]  post.bom
│   ├── [ 34K]  pre.bom
│   └── [356M]  usr
│       └── [356M]  standalone
│           └── [356M]  update
│               └── [356M]  ramdisk
│                   ├── [148M]  arm64eSURamDisk.dmg
│                   ├── [1.0K]  x86_64SURamDisk.chunklist
│                   └── [208M]  x86_64SURamDisk.dmg
├── [5.3K]  Info.plist
└── [ 232]  META-INF
    └── [ 178]  com.apple.ZipMetadata.plist

  48G used in 248 directories, 3026 files

But it doesn’t boot :(

InstallAssistant.pkg (XAR)
- DMG (compressed disk image)
  - ZIP
    - DMG (compressed disk image)
      - OTA payload / AppleArchive format / pbzx?
        
        ???

Source: ChatGPT 4 (DALL-E 2) trained on art stolen from the Internet

Create install media

Install to /Applications:

installer -pkg InstallAssistant.pkg -target LocalSystem

Create and mount an empty disk image:

hdiutil create -o ./image -size 15g -fs 'HFS+J'

hdiutil attach ./image.dmg -noverify -mountpoint ./mnt

Create install media:

/Applications/Install macOS Sonoma/Contents/Resources/createinstallmedia --nointeraction --volume ./mnt

This must run on macOS :(

The cat hasn’t found a way to install and set up MacOS without using GUI.

CLI is no go

Source: ChatGPT 4 (DALL-E 2) trained on art stolen from the Internet

How to automate GUI?

keyboard/mouse input

screenshot

black box

Controlled system

xdotool, vncdotool, dotool (Wayland, X11, TTY), …

CV backends:

Tesseract OCR (text matching)
OpenCV (template, feature, haar cascade, template-feature, and mixed matching)
…

Control backends:

xdotool (X.org)
vncdotool (VNC)
…

guibot

A tool for GUI automation using a variety of computer vision and display control backends.

https://github.com/intra2net/guibot

Apache Guacamole & OpenNebula

VNC/RDP/other

Web Browser

Source: https://guacamole.apache.org/doc/gug/guacamole-architecture.html

Guacamole dotool

https://github.com/jirutka/guacamole-dotool/

guacamole-dotool --url wss://example.org/guacamole -- \
  key Cmd+t pause 2 type 'echo "Hello, world!"' \
  key Enter pause 0.5 capture screenshot.png

import { Client, setDomGlobals } from 'guacamole-dotool'

setDomGlobals()
const c = await Client.connect('wss://example.org/guacamole')

await c.keysPress('Cmd', 't')
await c.pause(2)
await c.type('echo "Hello, world!"')
await c.keysPress('Enter')
await c.pause(0.5)
await c.captureScreenToFile('screenshot.png')

c.disconnect()

CLI

CI Pipeline

installer-image-fetch

opencore-image-fetch

check-images

setup

customize

finalize

install

create and start VM

run guibot script

bot.wait(Text('Restore from Time Machine'), 300)

Wait for the macOS Recovery image to boot

Open Terminal

18:19:14 Connecting to one.example.org
18:19:15 Waiting for the macOS Recovery image to boot
18:19:15 Waiting for Restore from Time Machine
18:20:49 Screen OCR:
╔═══════════════════════════════════════════════════════════════════════════════
‖ Utilities  Window
‖ 
‖ Restore from Time Machine
‖ 1f you have backup of your system that you want o restore.
‖ 
‖ Install macOS Sonoma
‖ Use the attached installer to Upgrade or Install macOS.
‖ 
‖ Safari
‖ Browse Apple Support to get help with your Mac.
‖ 
‖ Utity
‖ 
‖ Repair or erase a disk using Disk Utiity.
‖ 
‖ Continue
╚═══════════════════════════════════════════════════════════════════════════════
18:20:49 Opening Terminal
18:20:49 Pressing together keys 'Shift'+'Super'+'t'
18:20:50 Waiting for Terminal

Open Terminal

bot.press_keys([keys.SHIFT, keys.SUPER, 't'])
bot.wait(Text('Terminal'), 5)

Increase font size

bot.type_text('+++++', [keys.SUPER])

Locate the target disk

bot.type_text(FIND_DISK_SCRIPT); bot.idle(15)
bot.press_keys(keys.ENTER)
bot.wait(Text('DISK FOUND'), LONG_WAIT)

for disk in $(diskutil list | grep -o '/dev/disk[0-9]*'); do
    info=$(diskutil info -plist $disk)

    disk_type=$(echo "$info" \
               | plutil -extract VirtualOrPhysical raw -)
    [ "$disk_type" != Virtual ] || continue

    content=$(echo "$info" | plutil -extract Content raw -)
    [ -z "$content" ] || continue

    TARGET_DISK="$disk"
    printf "\\n\\n\\nDISK FOUND\\n\\n\\n"
    break
done

Format the target disk

bot.type_text(FORMAT_DISK_SCRIPT); bot.idle(4)
bot.press_keys(keys.ENTER)
bot.wait(Text('DISK FORMATTED'), LONG_WAIT)

diskutil eraseDisk APFS System $TARGET_DISK \
    && printf "\\n\\n\\nDISK FORMATTED\\n\\n\\n"

Start macOS installer

bot.type_text(...)
bot.press_keys(keys.ENTER)
bot.find_text_by_regex('Preparing:')

cd /Volumes/"Image Volume"/Install*app/Contents/Resources
./startosinstall --agreetolicense --volume /Volumes/System

Rebooting

Installing…

Rebooting again

And again…

After 30 min since the start…

Source: ChatGPT 4 (DALL-E 2) trained on art stolen from the Internet

Setup Assistant

bot.wait(Text('Select Your Country or Region'), 90 * 60)

CI Pipeline

installer-image-fetch

opencore-image-fetch

check-images

setup

customize

finalize

install

Select United States

bot.type_text('United States')
bot.click(Text('United States'))

Find and click on Continue button

right_button = bot.click(Button('Continue'))

Click on Continue

bot.wait(Text('Written and Spoken Languages'), LONG_WAIT)
bot.click(right_button)

Click on Continue Not Now

bot.wait(Text('Accessibility'), LONG_WAIT)
bot.click(right_button)

Click on Continue

bot.wait(Text('Data & Privacy'), LONG_WAIT)
bot.click(right_button)

Find and click on “Not now” button link

bot.wait(Text('Migration Assistant'), LONG_WAIT)
bot.click(Text('Not now'))

Find and click on “Set Up Later” link

bot.wait(Text('Sign In with Your Apple ID'), LONG_WAIT)
bot.click(Text('Set Up Later'))

Click on Skip

if bot.exists(Text('Are you sure you want to skip'), 5):
bot.click(Button('Skip'))

Click on Agree

bot.wait(Text('Terms and Conditions'), LONG_WAIT)
bot.click(right_button)

Click on Agree… again

bot.wait(Button('Disagree'), 10)
bot.click(Button('Agree'))

Fill out the form

bot.wait(Text('Create a Computer Account'), LONG_WAIT)
bot.type_text('Admin')  # Full name
bot.press_keys(keys.TAB)
bot.idle(0.5)
bot.type_text('admin')  # Account name
bot.press_keys(keys.TAB)
bot.idle(0.5)
bot.type_text(ADMIN_PASSWORD)  # Password
bot.press_keys(keys.TAB)
if bot.exists(Text('Password requirements'), 2):
    raise ValueError('Too weak password')
bot.type_text(ADMIN_PASSWORD)  # verify
bot.idle(1)
bot.click(right_button)  # Continue

Click on Continue

bot.wait(Text('Enable Location Services on this Mac'), LONG_WAIT * 4)
bot.click(right_button)

Find and click on “Don’t Use” button

bot.click(bot.wait(Button("Don't Use"), LONG_WAIT))

Select timezone Prague

bot.wait(Text('Select Your Time Zone'), LONG_WAIT)
bot.press_keys(keys.TAB)
bot.type_text('Prague')

Uncheck Share Mac Analytics with Apple

bot.wait(Text('Analytics'), LONG_WAIT)
bot.click(Text('Share Mac Analytics with Apple'))
bot.idle(SHORT_WAIT)

Find and click on “Set Up Later” link

bot.wait(Text('Screen Time'), LONG_WAIT)
bot.click(Text('Set Up Later'))

Click on Continue

bot.wait(Text('Choose Your Look'), LONG_WAIT)
bot.click(right_button)

Setup Assistant completed!

log.info('Waiting for Setup Assistant to complete')
bot.wait(Text('Window'), LONG_WAIT * 3)

Enable Full Disk Access for Terminal.app

Needed to adjust specific system settings from Terminal (e.g. enable remote SSH access).

It’s designed so that only a user can enable it, not a script.

Tab navigation

Ctrl + F7

Switch Tab to navigation through all controls on the screen, including OK/Cancel buttons.

Open System Settings

# Move focus to the menu bar.
bot.press_keys([keys.CTRL, keys.F2])
bot.idle(0.5)
bot.press_keys(' ')  # open the Apple logo item
bot.idle(0.5)
if MACOS_VERSION < 14:  # Ventura and older
    bot.press_keys(keys.DOWN)  # select About This Mac
bot.press_keys(keys.DOWN)  # select System Settings…
bot.press_keys(keys.ENTER)

Search for “Full Disk Access”

bot.press_keys([keys.SUPER, 'f'])  # move focus to the search bar
bot.idle(1)
# XXX: Search sometimes doesn't work on Sonoma.
bot.type_text('full disk ac')
bot.wait(Text('Privacy & Security'), LONG_WAIT)
bot.idle(SHORT_WAIT)
bot.press_keys(keys.DOWN)  # select Privacy & Security
bot.idle(SHORT_WAIT)  # we have to wait till it's loaded in the right panel
bot.press_keys(keys.DOWN)  # select Allow applications to access all user files

# Title in the main part of the System Settings window.
bot.wait(Text('Full Disk Access'), LONG_WAIT)
bot.idle(1)

Click on [+]

bot.press_keys(keys.TAB)
bot.press_keys(keys.TAB)  # [+]
bot.press_keys(' ')

Enter admin password

bot.wait(Button('Modify Settings'), LONG_WAIT)
bot.type_text(ADMIN_PASSWORD)
bot.press_keys(keys.ENTER)

Search for Terminal

bot.wait(Text('Documents'), LONG_WAIT)
bot.press_keys('/')  # invoke Go to the folder
bot.type_text('Applications/Utilities/Terminal')
bot.press_keys(keys.ENTER)

Wait for Open button and press Enter

bot.wait(Button('Open'), SHORT_WAIT * 2)
bot.press_keys(keys.ENTER)
bot.wait_vanish(Button('Open'), LONG_WAIT)

Open Terminal

bot.press_keys([keys.SUPER, ' '])
bot.press_keys(keys.BACKSPACE)
bot.type_text('Terminal')
bot.press_keys(keys.ENTER)

Open Terminal

bot.type_text('printf "\\n\\n\\nREADY\\n\\n\\n"')
bot.press_keys(keys.ENTER)
bot.wait(Text('READY'), LONG_WAIT)

Enable sudo, SSH access and print IP

bot.type_text('sudo -i')
bot.press_keys(keys.ENTER)
bot.type_text(ADMIN_PASSWORD)
bot.press_keys(keys.ENTER)

bot.type_text("echo 'admin ALL = (root) NOPASSWD: ALL' | tee /etc/sudoers.d/admin")
bot.idle(4)
bot.press_keys(keys.ENTER)

bot.type_text('systemsetup -setremotelogin on && echo printf "\\n\\n\\nDONE\\n\\n\\n"')
bot.idle(4)
bot.press_keys(keys.ENTER)
bot.wait(Text('DONE'), LONG_WAIT)

bot.type_text('clear')
bot.press_keys(keys.ENTER)
bot.type_text("""
ifconfig en0 inet | sed -En 's/^[[:space:]]inet ([^ ]+).*/\\n\\n\\nIP=\\1\\n\\n\\n/p'
""")
bot.idle(1)
bot.press_keys(keys.ENTER)

CI Pipeline

installer-image-fetch

opencore-image-fetch

check-images

setup

customise

finalise

install

Using shell script via SSH:

Copy EFI from OpenCore to the target disk
Disable automatic updates
Disable sleep
Set 24-hour format
Enable autologin as user admin
Install XCode
Install Homebrew or MacPorts
Install macos-init
Install additional packages
Cleanup caches

Customise

>150minutes

From 0 to customised bootable VM image with

Notes

I used OpenNebula because it was requested, but I don’t recommend it.
macOS GUI is unreliable and non-deterministic (under load).
I considered using VoiceOver for automation, but it was no better.

I haven’t fully explored all possible approaches due to time constraints.
If you know about a better way, please tell me after the talk.

Q&A

Source: ChatGPT 4 (DALL-E 2) trained on art stolen from the Internet

Automated macOS installation on VMs

By Jakub J.

Automated macOS installation on VMs

How to install macOS on a Linux virtualisation infrastructure – from downloading the installer from Apple, through installation, initial setup, customisation and creating a bootable disk image. And it’s done in a way that Cupertino probably hasn’t even thought of – without the use of a mouse, fully automated with CI. For comparison, I’ll also show how easy it is to do the same thing with Alpine Linux.

Jakub J.

github.com/jirutka