Links for The Past, Present and Future of Munki

Here are the URLs for many of the things I talk about in my presentation on Tuesday, May 23 in Brighton, UK.



Munki-Dev Google Group:
Munki-Discuss Google Group:

Munki Client Cert Authentication:

Munki Web Request Middleware:

Munki Authorized Restarts:





Staging macOS Insstallers:

Munki Default Installs:

Mac Admins Open Source:

Signed and Notarized builds of Munki:

Links for The Past, Present and Future of Munki

Radmind: converting to case-insensitive transcripts

Radmind, being a set of UNIX tools, originally supported only case-sensitive transcripts. Mac OS X’s HFS+ filesystem, developed by Apple pre-NeXT purchase, is a case-preserving, case-insensitive filesystem.

Support for case-insensitive transcripts was later added to the radmind tools.

As it turns out, it is perfectly possible to use radmind with case-sensitive transcripts to manage an OS X HFS+ filesystem. There are sometimes a few annoyances, but it generally works OK. Worst case, you might have to run the radmind tools twice to get the filesystem update when there is a case change: the first run might remove the lowercase version of the file, and the second run would install the uppercase version. Or a sharp radmind admin might be able to avoid the problem altogether by renaming files in troublesome transcripts.
Continue reading “Radmind: converting to case-insensitive transcripts”

Radmind: converting to case-insensitive transcripts

Radmind, 10.4.8, and Intel

Intel Core DuoQuite a while back, I did a series of posts on using radmind to update a machine from 10.3.x to 10.4.x. A major element of the strategy, developed by Andrew Mortenson, was to copy vital tools and their needed libraries to a “cache” directory and coerce the OS to use those copies instead. This worked around problems to reared their heads when radmind replaced those tools and libraries while it was updating the filesystem.

While working on building a radmind loadset for 10.4.8, I discovered that if I used my current script to have radmind update an Intel Mac from 10.4.7 to 10.4.8, that the machine would hang when the script attempted to reboot the machine. You could manually reboot it, and the machine would come up fine. This behavior seemed very similar to what happened when trying to go from 10.3.x to 10.4.x before I implemented Andrew’s technique. It seemed that Andrew’s technique didn’t go quite far enough. There was much discussion of the issue on the radmind-users mailing list, and Ian Ward Comfort of Stanford came up with a solution. It takes Andrew’s ideas, and builds on them in two ways:

  1. It dynamically determines if any tools or required libraries need to be cached, using Apple’s otool tool.
  2. If /usr/lib/dyld is being replaced, it calls all the tools using a chroot-ed environment so that the cached version of dyld is used. This was the key change for the 10.4.7 to 10.4.8 update on Intel – without the chroot-ed environment, the OS was calling dyld in its original location after radmind had swapped it out, leading to a crash.

In my testing, this technique works perfectly. Here is Ian’s script, and here’s a link to the tool-caching bit of code. I did a lot of hacking to meld Ian’s code with mine, and I’ll probably post my version of the script soon.

Radmind, 10.4.8, and Intel

Radmind and mountpoints

radmind iconYesterday we had a near-disaster where another admin created a mountpoint at /mnt and mounted an NFS filesystem for testing. After testing, he proceeded to run the radmind tools to update his machine. He noticed that radmind was scanning the contents of /mnt and aborted the run. If he had not aborted it, it’s likely that lapply would have started deleting items on that mount.

In the dim, dark past I asked for a feature for fsdiff that would cause it to ignore remotely mounted filesystems to prevent just this problem, having seen it crop up a few times here, usually due to an error.

Normally this is not a problem, as things like AFP and SMB servers normally mount under paths that are already in a negative transcript – places like /Network, /Volumes, and /automount. But a user with sudo rights (or an admin) can create mountpoints virtually anywhere and mount things. A more likely scenario that can lead to this type of problem is implementing classic UNIX NFS automounts like /home – a radmind administrator’s mistake in a transcript could lead to lapply trying to remove the contents of /home!

When this almost happened again yesterday, I had the flash of insight that I could script around the problem, and I came up with the attached script.
Continue reading “Radmind and mountpoints”

Radmind and mountpoints

Certificate creation for radmind

radmind iconRadmind supports the use of OpenSSL certificates as a means of identifying clients and for encrypting communications between client and server. I don’t really care about the encryption, but using certificates to identify clients allows laptops to run radmind no matter what network they use to connect to my radmind server – Ethernet, wireless, or VPN from home.

Setting up certificate support on a radmind server is a bit of a pain. I used the “Using TLS with Radmind” document available here.

But adding and removing certificates involves a lot of steps, and therefore is tedious and error-prone. And I wanted to be able to let several people at my organization be able to do this. When faced with repetitive, tedious, error-prone tasks, the lazy admin will write a script to do most of the work for him/her. So that’s what I did. Continue reading “Certificate creation for radmind”

Certificate creation for radmind

Radmind 1.6, Tiger and symlinks

radmind iconEarlier this month, I posted about an issue with radmind 1.5.1 when used under Tiger — symlinks could be created in such a way that they were unusable by anyone but root. This causes lots of other problems.

This was actually caused by a combination of a change in radmind to make the creation of temporary files more secure, and a difference in how symlinks behave in Tiger vs just about every other UNIX in existence.

radmind 1.6 was released this week, and contains changes that prevent this issue from happening under Tiger. Add in the new support for optional network compression and Universal Binary goodness, and you should definitely check it out.

Note that radmind 1.6.1 was released shortly after the release of 1.6. The only change was a fix for a compile problem on non-OS X OSes. There is a Mac OS X installer for 1.6; with 1.6.1 you have to compile from source. When I tried compiling 1.6.1, I got platform-specific binaries instead of Universal Binaries. So I didn’t spend any more time on the issue, and grabbed the 1.6 release instead.

Radmind 1.6, Tiger and symlinks

Universal OS X now!

Andrew Mortensen of the University of Michigan has documented a universal build of OS X – a single image that will boot both PPC Macs and Intel Macs. More here.

Combine this info with Jonathan Retzsch’s work on booting Intel Macs from APM-formated FireWire drives, and you should be able to create a FireWire drive that boots both Intel and PPC Macs without needing multiple boot partitions.

I wonder if this can be extended to build a single NetBoot image that works cross-platform…

Universal OS X now!

radmind 1.5.1, Tiger, and symlinks

A little over a month ago, I discovered an issue with the way radmind 1.5.1 created symlinks under Tiger. This exhibited itself as a broken symlink to a directory on an automounted NFS volume. After discussing the matter with the radmind developers, and being unable to demonstrate any other actual problems caused by this, I decided to script a fix to the broken symlink and move on.

Yesterday, James Reynolds posted to both the MacEnterprise list and the radmind-users list about a problem with Adobe Bridge crashing on launch. James had used ktrace to discover it was crashing when trying to read a symlink.

James also discovered that if he copied the offending files from another (good) machine, Bridge worked as expected. It was only when radmind installed the files that things broke.

This rang a bell. I was quickly able to duplicate James’ issue by using radmind 1.5.1 to install Adobe Photoshop CS2 on a Mac.

After recreating the symlinks like so:

grep -h "^l" /var/radmind/client/AdobePhotoshopCS2.T | /usr/local/bin/lapply -u 022

Adobe Bridge worked as expected.

What is going on here?

There is a bug in Tiger that radmind 1.5.1 is exercising. Classically, owner, group and mode should be ignored on symlinks, deferring to the owner, group and mode of the symlink’s target. But in some operations, Tiger seems to honor symlink permissions, and if they are set so that a user cannot read the link, access to the link’s target will fail.

Here is a demonstration of the bug outside of the context of radmind:

(istanbul.fas) root [210] % uname -a
Darwin 8.5.0 Darwin Kernel Version 8.5.0: Sun Jan 22 10:38:46 PST 2006; root:xnu-792.6.61.obj~1/RELEASE_PPC Power Macintosh powerpc
(istanbul.fas) root [211] % cd /
(istanbul.fas) root [212] % umask 077
(istanbul.fas) root [213] % ln -s /Users/Shared /Shared
(istanbul.fas) root [214] % ls -al /Shared
lrwx------ 1 root admin 13 Mar 2 10:33 /Shared@ -> /Users/Shared

Note the restricted permissons for the /Shared symlink.

(istanbul.fas) root [215] % umask 022
(istanbul.fas) root [216] % readlink /Shared
(istanbul.fas) root [217] % exit
(istanbul.fas) gneagle [202] % readlink /Shared
(Nothing is returned)

As root, “readlink /Shared” returns the target path: “/Users/Shared”.
As “gneagle”, a non-privileged user, it returns nothing.

radmind 1.5.1 changed the way it creates files in an attempt to improve security. It changes the umask to 077, creates the file, and then chmods it to its intended permissions. But chmod has no effect on symlinks, and radmind captures no user, group or mode info for symlinks since the are not supposed to be significant.

The net effect is that all sysmlinks are created with owner as root and with mode 700. Normally that would not be a problem, and indeed is not under Panther. But under Tiger, it can be a problem.

So until Apple fixes the bug (I’ve reported to them) or the radmind team implements a workaround in their code, you can avoid this issue going foward by calling lapply with a “-u 022” option. This tells lapply to use a umask of 022 when creating temporary files. The net effect is that symlinks are created with mode 755, which makes them usable by non-root users.

That still leaves a major problem to solve. How do you fix the symlinks already created by radmind with a umask of 077? I developed a script to do just that.

#!/usr/bin/perl –w
# by Greg Neagle,
# - This script attempts to fix broken symlinks created by
# lapply 1.5.1 when run with the default umask option
# - No attempt is made to fix symlinks referenced in negative
# transcripts
# - Symlinks that are referenced in a positive transcript and a
# higher-precedence negative transcript therefore _will_ be
# modified, this could be bad. In my loadsets, I can find no
# actual occurrences of this problem, since the only symlink I
# have in any negative transcript is this one:
# l /private/etc/localtime /usr/share/zoneinfo/US/Pacific
# and it doesn't appear in any other transcripts.
# But the danger is there.
# - A deeper potential problem is transcript entries that
# _remove_ symlinks: if a transcript creates a symlink, and
# a higher-precedence transcript removes it, we'll have an
# unwanted symlink hanging around. A subsequent radmind
# session will remove it, but if this script is run again,
# it will come back.
# - Therefore we need to deal with "- l" lines in the
# transcripts as well. Unfortunately, we cannot just blindly
# pass these lines to lapply, because if symlink doesn't exist
# in the filesystem, lapply will fail and stop processing
# transcript lines.
# So we'll need to feed symlink removal lines to lapply
# one-by-one and ignore failures.
use strict;
my $radmindClientDir = "/var/radmind/client";
my $commandFile = $radmindClientDir . "/command.K";
my $symlinkremovet = "/tmp/symlinkremove.T";
#turn off output buffering
(open K, $commandFile) or die "Can't open the command file!";
while (<K>) {
   my $commandFileLine = $_;
   if ($commandFileLine =~ /^p/) {
      #positive transcript
      my @lineItems = split;
      my $transcript = $lineItems[1];
      print "Processing $transcript...\n";
      # re-create all symlinks in the transcript:
      # grep out all lines that begin with l and send them
      # to lapply with a 022 umask
      system "/usr/bin/grep -h \"^l\" \"$radmindClientDir/$transcript\" | /usr/local/bin/lapply -u 022";
      # grab symlink removal lines
      system "/usr/bin/grep -h \"^- l\" \"$radmindClientDir/$transcript\" > $symlinkremovet";
      print "Removing symlinks flagged for removal in $transcript...\n";
      if (open T, $symlinkremovet) {
         while (<T>) {
            #feed each line one-by-one to lapply
            system "echo $_ | /usr/local/bin/lapply -u 022";
         close T;
close K;

radmind 1.5.1, Tiger, and symlinks