Saturday, March 14, 2015

Dual Root Upgrades

Performing operating system upgrades is always a risky task. Obvious issues like unrecognized hardware and incompatible application software. Less obvious risks like subtle bugs or incompatibility with support systems that might not be noted right away. In production environments these problems can lead to extended downtime and failure to stay within published maintenance windows. The dual-root architecture I developed to try and reduce these risks with a very simple recovery process.

The technique may be applied to most operating systems that run on general purpose computers. The remainder of this journal entry will summarize how I have chosen to implement it on FreeBSD.

Setup

  1. In addition to the normal filesystem partitioning scheme, add a second partition that is an identical size to the planned root partition. 
    • This must cover / and /boot and 
    • should cover /usr but 
    • may not need to cover /usr/local. 
    • Ideally it should not cover /usr/src or /usr/ports. 
    • It must not cover /home or /usr/home or any other user or application data directories.
  2. Tell the installer to use an alternate location for the root filesystem when upgrading.
    • for source this is done with the INSTALLROOT variable/option.
    • for bsdinstall do a custom install with the new root partition mounted on /mnt.
For reference when defining the size of the root partition(s):
  • FreeBSD 10.1-RELEASE base install from binary used 1.4GB including the ports skeleton
  • FreeBSD 9.3-RELEASE-p10 base, source, ports, and several packages used 3.5GB before compiling the source tree.
  • I currently use 16GB for each root partition. I would reduce this to 8GB each if I shared /usr/src and /usr/ports between the two.
  • In a production environment, I would probably share /usr/local as well to allow for separate upgrading of OS and applications.

Selecting the Root Partition

The FreeBSD boot loader (boot0) scans the partitions in order looking for /etc/fstab and assumes that is the root partition by default. At least, that is what is (or was) in the source code comments. Fortunately for us, the gpart utility allows us to set some attributes that override that default behaviour.

If we assume that our two root partitions are located on /dev/ada0p2 and /dev/ada0p3, then the boot loader will use /dev/ada0p2 by default all the time. To tell the boot loader to use /dev/ada0p3 we need to set a special attribute with the gpart utility:

gpart set -i 3 -a bootme /dev/ada0

and to unset it:

gpart unset -i 3 -a bootme /dev/ada0

for clarity sake, you may want to set the bootme attribute on the first partition. However, the boot loader will boot the first available partition that appears to be a root partition by default.

No comments:

Post a Comment