DIY Docker on Apple silicon M1

When Apple announced the transition to Apple's own ARM-based silicon, I was ecstatic! I've always enjoyed tinkering with ARM based single-board computers such as the raspberry pi 4 and Pinebook Pro. But they always had sub-par performance and here was Apple trying to transition their entire lineup!

Even though we all knew the performance should be good based on how recent iPad Pros scored, I don't think many expected the M1 chip, Apple's first iteration, to then beat Intel's most high-end CPUs while redefining "all-day battery life"!

So naturally, I found myself clicking buy on day 1, something I rarely do before doing a ton of research. I ended up ordering the base Macbook Air but bumping up the RAM to 16GB in order to better run Docker.

Disappointment: Docker support not ready

My machine arrived quickly, despite getting shipped directly from China as a result of my memory upgrade. The first step was to setup the environment just the way I liked using InstallBuddy, a script I wrote a while back to help with this sort of thing. And there hit my first snag, as I discovered that homebrew didn't yet have native support. After jumping through few hoops to get homebrew setup via Rosetta 2, and my goto packages installed, I was on my way to downloading Docker Desktop for Mac.

I naively expected Docker to "just work" as it was a touted feature, briefly mentioned during Apple's transition announcement and then later followed up in more detail during the AppleInsider interview. Unfortunately, I soon came across an official docker blog post about Docker Desktop not being quite ready but was being worked on.

Hope: Apple support for Linux virtualization

Impatient, I searched on and came across Apple's developer documentation for supporting Linux through virtualization. This was promising and I knew it was only a matter of time before someone built an Objective C or Swift wrapper around it.

A few days later, I came across this well written article: Running Docker on Apple Silicon M1 which had a nice step-by-step tutorial to getting it setup using a project called vftool and the ARM64 version of Ubuntu 20.04 image. Following those steps, I was able to get docker running, albeit in a read-only live cd that had a small COW filesystem. The 2nd follow up article has steps to show how to connect to the docker daemon from docker cli running on macOS, thus giving something closer to what Docker Desktop for Mac had to offer.

Contribution: Getting full read/write support with data persistence

Some of the downsides to running off the livecd was that the default COW filesystem can quickly run out of diskspace as it was only about 1GB in size. As the COW filesystem is merely an overlay on top of a read-only squashfs filesystem, uninstalling packages won't help (in fact will eat into the COW space). Another downside was that in order to get Docker working properly, vfs filesystem driver needed to be configured and this was known to be less performant.

Therefore I wanted to see if there was a way to install Ubuntu onto a read-write filesystem. Below are the steps I followed in order to achieve that. Please first read the above linked post to get vftool setup and kernel and initrd images extracted from the live CD iso from within macOS.

Below is how my directories are structured, so you will need to adapt to your situation.

Steps 1: Create a loopback disk image

After testing a few disk sizes, I settled on a 10GB loopback disk image.

dd if=/dev/zero of=disk.img bs=1m count=10240

Step 2: Create a temporary VM to install Ubuntu

The next step is to boot into a VM similar to how the article I've linked explains. We use this VM to then manually install Ubuntu onto the loopback disk.img

./vftool -k ./vmlinuz -i ./initrd-livecd -c ./focal-desktop-arm64.iso -d disk.img -m 4096 -a "console=hvc0"

On another terminal connect to the VM

screen /dev/ttys002

Step 3: Partition and mount the loopback disk

The disk should be mounted on to /dev/vda

cfdisk /dev/vda

Partition type: gpt
/dev/vd1 Linux filesystem

Save and exit. Then format the partition as ext4

mkfs.ext4 /dev/vda1

Step 4: Mount new partition and copy files to it

Mount and start copying files from /rofs which is where Ubuntu mounts the read-only squashfs file system.

mount /dev/vd1 /mnt
cd /rofs
cp -axv . /mnt/

Step 5: Chroot onto loopback filesystem to perform post install tasks

mount -t proc none /mnt/proc
mount -t sysfs sysfs /mnt/sys
mount -o bind /dev /mnt/dev
chroot /mnt

Next lets set the /etc/fstab so it knows how to mount /

editor /etc/fstab
# UNCONFIGURED FSTAB FOR BASE SYSTEM 0.0 0 0.0
0
UTC dpkg-reconfigure tzdata

Let's also create a user for us to login and be able to sudo.

adduser bud
addgroup --system admin
adduser bud admin

Step 6: Making a new initrd

The initrd we extracted from the livecd is not suitable for booting off the loopback disk.img so we need to generate a new one. Still within the chrooted environment run the following:

mkinitramfs -c gzip -o /boot/initrd.img-$(uname -r)

Next we need to copy the newly generated initrd.img back onto our macOS filesystem. I did this by enabling SSH within macOS.

scp /boot/initrd.img-$(uname -r)
bud@192.168.86.28:/Downloads

Then copied the file from Downloads to where the other files were.

Step 7: Exit and unmount from chrooted environment

exit
umount /mnt/root/sys
umount /mnt/root/proc
umount /mnt/root/dev
umount /mnt

Then shutdown the VM properly by issuing a poweroff

sudo poweroff

Step 8: Ready to launch the new persistent VM

We can now get rid of the cdrom image and boot directly on to the disk.img using the same kernel and new initrd.img-5.4.0-56-generic image.

./vftool -k ./vmlinuz -i ./initrd.img-5.4.0-56-generic -d disk.img -m 4096 -a "console=hvc0 root=/dev/vda1"

From another terminal...

screen /dev/ttys002

We should now be able to login using the account you created within the chrooted environment.

Step 9: Setting up SSH

It's far easier to work within an SSH session that within the screen session and SSH can also be used to link docker cli running on macOS to connect to the daemon running inside the VM (more on this later)

sudo apt install openssh-server

Obtain the IP address within the VM and try connecting to it.

ip addr show dev enp0s1 | grep 'inet '
ssh bud@192.168.64.13

Copy your macOS account's SSH key to the remote VM in order to enable password-less login, which is also needed by docker cli when connecting from the Mac.

ssh-copy-id bud@192.168.64.13

Step 10: Setting up Docker inside VM

These steps are copied from the linked article which are adapted from docker's official installation guide.

sudo apt-get install apt-transport-https ca-certificates \
curl gnupg-agent software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository \
"deb [arch=arm64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

To make it easy to run docker commands as the user I added myself to the docker group.

sudo usermod -aG docker ${USER}
su - ${USER}

Now logout and log back in for the new group to take effect. Docker should now be running properly.

docker run --rm hello-world

Step 11: Setting up Docker cli to connect to VM

For a near Docker for Mac like experience, we need to install docker cli on macOS. Unfortunately at the time of writing, this needs to be installed via Rosetta 2. I used homebrew to accomplish this, but you could also just download the binary directly from Docker Inc.

arch -x86_64 brew install docker

We then create a docker context and switch to it to seamlessly connect to the docker daemon running inside the VM.

docker context create myvm --docker "host=ssh://bud@192.168.64.13"
docker context use myvm
docker run --rm hello-world

You should now be able to issue docker commands from within macOS.

Step 12: Uninstall gnome desktop (optional)

Finally, I got rid of gnome desktop to reclaim more space as everything is running via the terminal. When trying this I ran into an issue with the flash-kernel package so you might want to first uninstall it.

sudo dpkg -r flash-kernel

sudo apt purge adwaita-icon-theme gedit-common gir1.2-gdm-1.0 \
gir1.2-gnomebluetooth-1.0 gir1.2-gnomedesktop-3.0 gir1.2-goa-1.0 \
gnome-accessibility-themes gnome-bluetooth gnome-calculator gnome-calendar \
gnome-characters gnome-control-center gnome-control-center-data \
gnome-control-center-faces gnome-desktop3-data \
gnome-font-viewer gnome-getting-started-docs gnome-getting-started-docs-ru \
gnome-initial-setup gnome-keyring gnome-keyring-pkcs11 gnome-logs \
gnome-mahjongg gnome-menus gnome-mines gnome-online-accounts \
gnome-power-manager gnome-screenshot gnome-session-bin gnome-session-canberra \
gnome-session-common gnome-settings-daemon gnome-settings-daemon-common \
gnome-shell gnome-shell-common gnome-shell-extension-appindicator \
gnome-shell-extension-desktop-icons gnome-shell-extension-ubuntu-dock \
gnome-startup-applications gnome-sudoku gnome-system-monitor gnome-terminal \
gnome-terminal-data gnome-themes-extra gnome-themes-extra-data gnome-todo \
gnome-todo-common gnome-user-docs gnome-user-docs-ru gnome-video-effects \
language-pack-gnome-en language-pack-gnome-en-base language-pack-gnome-ru \
language-pack-gnome-ru-base language-selector-gnome libgail18 libgail18 \
libgail-common libgail-common libgnome-autoar-0-0 libgnome-bluetooth13 \
libgnome-desktop-3-19 libgnome-games-support-1-3 libgnome-games-support-common \
libgnomekbd8 libgnomekbd-common libgnome-menu-3-0 libgnome-todo libgoa-1.0-0b \
libgoa-1.0-common libpam-gnome-keyring libsoup-gnome2.4-1 libsoup-gnome2.4-1 \
nautilus-extension-gnome-terminal pinentry-gnome3 yaru-theme-gnome-shell
sudo apt autopurge

Conclusion

This was a fun exercise and if you're impatient waiting for official support from Docker Inc. or just want to try it for the fun of it, go ahead. Maybe you can extend the setup and tackle other problems I've yet to encounter. Else it might be best to just wait a bit longer as official support will be coming sooner rather than later.

Location: Lakefront Trail, Chicago, IL 60657, USA

Comments

Post a Comment

Popular posts from this blog

Why KDE4 (might) suck!

Image
I've been a kde user for very long time, since from before version 3.0. I vaguely remember being excited with version 3.0 to the point I was using the beta releases on a daily basis. Similarly I have been anticipating kde 4.0, the next major release since 3.0 I suppose. But it seems that is just its problem! Its too much of a major release to be pushing out in such a hurry. I have been following kde 4 from time to time by reading some of the discussions on the panel-devel list, trying out the alpha releases via the SUSE live cd and of course discussing with Siraj , our local kde developer, on the internals (technology and community wise). While things have improved a bit, with each release of kde 4, its far from ready. We are now at RC1 and the announcement says its ready for prime time testing. Building on this, the majority of applications included in KDE 4.0 are now usable for day to day use. The KDE Release Team has recently underlined this by calling on the community to par...
Read more

Track Your New Year Resolutions With ii.do

Image
I know, I know, it’s a bit of a cheesy title to promote my Open Source project but here me out - it really does work. Though I didn’t really write it to track my new year resolutions (I’ve never found them effective beyond a week), I did write it out of constant frustration with not finding a TODO productivity tool that stuck with me. Let me explain… I’ve tried EVERYTHING! Google Calendar & tasks Sending myself Emails Wiki (Dokuwiki, Mediwiki, Twiki etc.) Tomboy Tomboy with UbuntuOne & Android App EverNote Remember the Milk Gedit notes on my desktop Post-it notes widget Actual Post-it notes Old-skool diary Pieces of paper including backs of payment receipts From the above list, if anything came remotely to sticking as a habit, then it’d be the sticky notes & pieces of paper. I’m not suggesting the other methods suck, but I wouldn’t use it beyond a couple of weeks. Why was it that despite me spending hours on a computer, the best thing that had a chance of r...
Read more

I want my CDMA! How my rights were violated.

Last Wednesday, I went to pick up a CDMA phone from Bell after hearing all good about it and how fast it was. I had been using Bell for now over 5 years, primarily to surf the Internet. Sure it wasn't fast at 33.6, but atleast it was the next best economical thing until ADSL shows up. Naturally when CDMA came along with a lower rental and a 5x claimed speed, it did raise my eye browse. But I had already invested 15,000/- on my current phone, which included paying the old owner his phone deposit of 7500 and another 7500 for the phone transfer. Still, I was paying 600+ on rental alone and wasn't using the two lines that came with it that much. After waiting and rethinking for about two months and checking on ADSL, I decided to bite the bullet and go for CDMA, while disconnecting the old line. Boy! was I in for a surprise as I left to the Bell head office It seemed as if I had committed the ultimate sin, by asking that they disconnect the old phone, while I buy another brand new ...
Read more