Revisiting receipts for OS X installs via Munki

Background

When using Munki to upgrade OS X with a package created using createOSXinstallPkg (https://github.com/munki/createOSXinstallPkg), the recommendation has been to change the auto-generated receipts array:

https://github.com/munki/munki/wiki/Installing%20OS%20X#receipts

The intention here was to provide something that would satisfy Munki’s check to see if an item needs to be installed: if the recommended receipt is present, Munki won’t attempt to (re-)install the package that installs OS X.

A problem

However, I recently discovered an issue with this approach. For some machines here, an “InstallYosemite” item was added to their manifest’s managed_installs some time ago to force an upgrade to Yosemite. Once that upgrade was complete, the normal installcheck mechanism found a receipt for “com.apple.pkg.BaseSystemBinaries” of version “10.10.0.1.1.1412852630” or higher, and did not offer to reinstall Yosemite, even though the item remained in “managed_installs” in that machine’s manifest.

Later we added “InstallElCapitan” as an optional_install for all users (in an included_manifest). If a user with a machine manifest like the one described above then chose to self-upgrade to El Capitan, as part of the El Capitan install, the “com.apple.pkg.BaseSystemBinaries” receipt is removed. (Note that at https://github.com/munki/munki/wiki/Installing-OS-X/17e17bfdc80727bbd83595359eec6db2741fe88c, the previous recommended receipt to check for El Capitan is “com.apple.pkg.Essentials” since the “com.apple.pkg.BaseSystemBinaries” is not present in an El Capitan install.)

Once the El Capitan upgrade is complete, when Munki later checks for updates, it encounters “InstallYosemite” in the managed_installs. Since the “com.apple.pkg.BaseSystemBinaries” receipt is no longer present, it decides it needs to install Yosemite once again. This of course, fails, since the package itself has an installcheck, which errors with:


Cannot install on volume / because it is disabled.
-------------------------------------------------------------------------
installer: Cannot install on volume / because it is disabled.
installer: You can’t upgrade this version of OS X because a newer version
is installed.
-------------------------------------------------------------------------

If your InstallElCapitan package also includes the Munki bootstrap flag (to ensure all other needed updates for El Capitan are performed), this can lead to a loop where Munki attempts to install Yosemite, fails, tries again, etc.

A fix

The obvious fix is to remove “InstallYosemite” from all manifests once you are offering “InstallElCapitan”. But I think I have a way to avoid this situation in the future. I will recommend a different installation check item for createOSXinstallPkg-type items.

Instead of modifying the auto-generated receipts array, I will now recommend adding an installs array. Here’s one for El Capitan:


<key>installs</key>
<array>
  <dict>
    <key>ProductVersion</key>
    <string>10.11.4</string>
    <key>path</key>
    <string>/System/Library/CoreServices/SystemVersion.plist</string>
    <key>type</key>
    <string>plist</string>
    <key>version_comparison_key</key>
    <string>ProductVersion</string>
  </dict>
</array>

This directs Munki to look at the ProductVersion key in /System/Library/CoreServices/SystemVersion.plist, and compare it with 10.11.4. (you could even just make it “10.11” to prevent Munki offering this as an upgrade to 10.11.[0-3] machines.)

/System/Library/CoreServices/SystemVersion.plist exists in all versions of OS X, so it’s a better comparison than package receipts that may or may not exist in future versions of OS X. And it’s easier to revise in the future since the ProductVersion maps to the OS X version you are installing.

Revisiting receipts for OS X installs via Munki

Packaging Lab at MacTech Conference 2015

If you are planning on participating in the Packaging Lab this week at the MacTech Conference, you may want to download some materials in advance when you aren’t competing with all the other people for limited conference Wi-Fi bandwidth. Note — don’t install these items — just download their installers and keep them handy for the lab.

ExifTool pkg:
http://www.sno.phy.queensu.ca/~phil/exiftool/ExifTool-10.03.dmg

Adobe Reader 11 pkg:
http://ardownload.adobe.com/pub/adobe/reader/mac/11.x/11.0.10/en_US/AdbeRdr11010_en_US.dmg

Google Earth pkg:
http://dl.google.com/earth/client/advanced/current/GoogleEarthMacNoUpdate-Intel.dmg

Firefox dmg:
http://ftp.mozilla.org/pub/firefox/releases/latest/mac/en-US/Firefox%2042.0.dmg

Google Chrome dmg:
https://dl.google.com/chrome/mac/stable/GGRO/googlechrome.dmg

Some optional things:

Packages:
http://s.sudre.free.fr/Software/files/Packages.dmg

Suspicious Package:
http://www.mothersruin.com/software/SuspiciousPackage/download.html

Pacifist:
https://www.charlessoft.com

Packaging Lab at MacTech Conference 2015

Pseudo-“Payload-free” pkgs with pkgbuild

“Payload-free” packages — that is, Apple installer packages that do not have a file payload, but only run scripts, are a nice tool for OS X admins to have. They provide a convenient way to deliver and execute scripts as root. If you have a way to install packages on your managed machines, you can also run scripts as root by wrapping them in a “payload-free” package.

Rich Trouton has written up the basic procedure using the built-in `pkgbuild` tool: https://derflounder.wordpress.com/2012/08/15/creating-payload-free-packages-with-pkgbuild/

But payload-free packages built this way have a “feature” that can sometimes prove problematic. Flat packages built with pkgbuild using the --nopayload option do not leave receipts in the pkgutil database. This means it can be difficult to determine if a given payload-free package has already been installed on a given machine.

This is especially annoying with Munki: by default, when installing a package, Munki uses the package’s receipt(s) to determine whether or not the package has been installed. Without that receipt, and with no other information, Munki can’t tell if the package has been installed.

Fortunately, it’s trivial to make a pseudo-payload-free package that leaves a receipt. All we need to do is specify an empty payload!

Here’s how we make a “true” payload-free package (that does not leave a receipt):

pkgbuild --nopayload --scripts /path/to/scripts_dir --identifier org.example.payloadfree --version 1.0 MyGreatPayloadFree.pkg

and here’s a “pseudo” payload-free package that does leave a receipt:

mkdir empty
pkgbuild --root empty --scripts /path/to/scripts_dir --identifier org.example.payloadfree --version 1.0 MyGreatPayloadFree.pkg

That’s it! Instead of using the --nopayload option, we create an empty directory and point the --root option at it. The package is built with the empty payload, and when installed, the package leaves a receipt.

Pseudo-“Payload-free” pkgs with pkgbuild

Setting up server-side resources for Imagr testing

Imagr requires a web server (or servers) to get its workflows, images to restore, and packages to install.

In an earlier post, I described setting up an existing DeployStudio NBI for Imagr testing.

If you are testing Imagr, or even better, hoping to help with its development, you’ll need to configure a test server.

You’ll need a web server, and a way to get some files on it. I just used my existing Munki repo web server. I created a new folder called “testing” and copied a few packages into it. (For first-boot install, Imagr currently supports only flat packages. For immediate install, bundle packages are OK if they are at the root of a disk image/.dmg file.).

Next, create a configuration plist. There’s an example here: https://github.com/grahamgilbert/imagr#the-configuration-plist. Here’s another:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>password</key>
  <string>40e78b0d11c553be61645796fdb9438271a23ed8b4aadb253a9222c6e4c861e2dbc75139000b0709fc0c5ad66b0f2573e38a72a8f629e270e59530fc3159d8e4</string>
  <key>workflows</key>
  <array>
    <dict>
      <key>name</key>
      <string>Munki Bootstrap</string>
      <key>description</key>
      <string>Installs the required pkgs to begin a Munki bootstrap on restart.</string>
      <key>components</key>
      <array>
        <dict>
            <key>type</key>
            <string>package</string>
            <key>url</key>
            <string>http://munki/repo/testing/DA_adminaccount_1.5.pkg</string>
            <key>pre_first_boot</key>
            <true/>
        </dict>
        <dict>
            <key>type</key>
            <string>package</string>
            <key>url</key>
            <string>http://munki/repo/testing/DisableSetupAssistant.pkg</string>
            <key>pre_first_boot</key>
            <true/>
        </dict>
        <dict>
            <key>type</key>
            <string>package</string>
            <key>url</key>
            <string>http://munki/repo/testing/munkitools-2.2.4.2431.pkg</string>
            <key>pre_first_boot</key>
            <true/>
        </dict>
        <dict>
            <key>type</key>
            <string>package</string>
            <key>url</key>
            <string>http://munki/repo/testing/munki_kickstart.pkg</string>
            <key>pre_first_boot</key>
            <true/>
        </dict>
      </array>
    </dict>
  </array>
</dict>
</plist>

The password is a hash for “LETMEIN”; you can certainly generate your own as explained here: https://github.com/grahamgilbert/imagr#password

The various URLs point to the pkgs I copied into the testing folder in my Munki repo share. Adjust as needed.

I saved this plist in the same folder as the testing pkgs as “testing.plist”. (Boring, I know.)
Make sure you can open it in a web browser before you continue — in my case it’s available
at http://munki/repo/testing/testing.plist

We are now done with the server config.

Boot a test client into the DeployStudio NBI that was configured as in my earlier post.

Open the Terminal.app.
We need to tell Imagr how to find the configuration plist, so:

defaults write com.grahamgilbert.Imagr serverurl http://munki/repo/testing/testing.plist

Make sure you replace the URL with the actual URL to the config plist you created.

Now launch Imagr:

/private/var/tmp/DSNetworkRepository/testing/Imagr.app/Contents/MacOS/Imagr

And bask in your success (or grumble at your failure…)

Setting up server-side resources for Imagr testing

Introducing Imagr

The prolific Graham Gilbert is working on a new tool: https://github.com/grahamgilbert/imagr

Imagr is an application designed to be run from a NetInstall environment. It is able to restore a disk image and install packages on a target volume.

Though Graham claims it is not intended to be a replacement for DeployStudio, I think in time it could very well be exactly that, at least for many people/organizations.

Some exciting things about Imagr:

  • It is open source! DeployStudio is free, but the source is closed. Open source means you can look at the code and see what it is doing. You can fix bugs and perhaps even help add features to the product.
  • It’s written in Cocoa-Python, with much of the “interesting” tasks in Python — a language many OS X admins are familiar with.
  • And most importantly: it does not require a specialized server: it can work with any plain-old web server. This means that you (potentially) can eliminate the last OS X device in your server room/data center. Almost every service of interest to OS X admins can be run on platforms other than OS X with the huge exception of a DeployStudio server. If Imagr can meet your imaging/machine build needs, you can finally get rid of that poor Mac mini sitting on a shelf in the server room.

Graham has posted detailed instructions on how to build a NetBoot image that contains Imagr. But if you just want to take a look and play with the tool, and you have an existing DeployStudio NBI with Python support included, you can play with Imagr without needing to build a new NBI. More importantly, you can test new versions (or work on the code yourself and test new versions you build) without having to build a new NBI each time.

Continue reading “Introducing Imagr”

Introducing Imagr

“Building” 2015 Macs

We’ve recently gotten a few of the new 2015 Macs, specifically the new MacBook Airs and 13″ MacBook Pros. These models have new builds of 10.10.2, and won’t boot from our current NetBoot/DeployStudio Server.

While there are ways of capturing the hardware-specific OS so you can use it with AutoDMG and to create NetBoot sets, usually what we do with new hardware like this is employ a “no-imaging” process: we boot the new machine in Target Disk Mode, attach it to another Mac, and install some “bootstrapping” packages on the new Mac. For us, that’s four packages:

  • A package to create a local admin account
  • A package to disable the Setup Assistant
  • The Munki tools package
  • A package to put Munki in “bootstrap mode” on first boot.

Once these packages are installed on the new machine, we shut it down, connect it to the network, and start it up. Munki takes over and installs everything else we need.

This approach has worked well for us in the past with new hardware, and allowed us to deploy new hardware to users much faster than if we had to build hardware-specific NetBoot and AutoDMG images.

But these new 2015 Macs were throwing us a curve ball. After using Target Disk Mode to install the packages, upon reboot the Macs were refusing to boot, or booting very slowly. A check with Disk Utility claimed the startup volume had unrepairable filesystem corruption, and advised us to back up any user data, format, and re-install.

We saw this behavior on multiple new 2015 machines. Two things that stood out to me:

  • The new machines all ship with an unencrypted CoreStorage volume as their main system volume, and
  • We were connecting these machines (in Target Disk Mode) to a Mac running Mavericks (10.9.5).

My theory was that there is a bug causing the Installer on Mavericks to do something “wrong” when installing to a Yosemite CoreStorage volume that led to file system corruption.

I did not have the time (or desire) to get to the bottom of this issue; I needed to find a successful procedure for getting these new 2015 Macs to build. I found two.

First one: after starting the new Mac in Target Disk Mode, connect to another Mac running OS X Yosemite 10.10.2, and install the bootstrapping packages.

Second one: Boot the new Mac into the Recovery partition. Connect a USB drive (a flash stick will do) containing the bootstrapping packages. Open the Terminal from the Utilities menu and use the command-line installer to install the bootstrapping packages onto the main partition.

With either method, things proceeded normally and there was no filesystem corruption.

UPDATE: after seeing some reactions to this post on Twitter, until we understand more about this issue, I’d recommend using the second approach (via Recovery partition) — it’s possible this is more about Thunderbolt/Target Disk Mode than it is about 10.9 vs 10.10.

I hope this saves others some time and headaches.

“Building” 2015 Macs

Accessing More Frameworks with Python

This post is based on a column I wrote for MacTech magazine in 2012. MacTech used to make older columns available online, but they haven’t done that for the past several years for some reason.
I’m planning to go through my older columns and dust off and republish some that I think are still relevant or useful.

Recently, we built a command-line tool using Python and the PyObjC bridge to control display mirroring.
PyObjC supports a lot of OS X frameworks “out-of-the-box”, and accessing them from Python can be as simple as:

include CoreFoundation

But what if the problem you want to solve requires a framework that isn’t included with the PyObjC bindings? In turns out that you can create your own bindings. In this post we’ll explore this aspect of working with Python and OS X frameworks.

OUR SAMPLE PROBLEM

In my organization, we sometimes have a need to set displays to a certain ColorSync profile. The ColorSync profile to use for a given display is a per-user preference, so if you need to set it for all users of a machine, you can’t just manually set it while logged in as one user and call it good.

If you are managing display profiles for a group of machines, or a conference room machine that has network logins, you need a way to manage display profiles for all users. Using MCX or doing some defaults scripting might come to mind. Let’s look at that possibility.

Continue reading “Accessing More Frameworks with Python”

Accessing More Frameworks with Python