Chapter 35. Updating an existing system from sources

Table of Contents

35.1. Manual build and update procedure
35.1.1. Building a new userland
35.1.2. Building a new kernel
35.1.3. Installing the kernel and userland
35.1.4. Updating the system configuration files
35.1.5. Summary
35.2. Using sysinst
35.3. Using sysbuild and sysupgrade
35.3.1. Tweak: Building as non-root
35.3.2. Tweak: Setting up nightly builds
35.4. More details about the updating of configuration and startup files
35.4.1. Using etcupdate with source files
35.4.2. Using etcupdate with binary distribution sets
35.4.3. Using etcmanage instead of etcupdate

A common mechanism for upgrading a NetBSD system to a newer version is by rebuilding the system from sources and installing the results. This works both for stable releases such as NetBSD 10.1 and for NetBSD-current. In particular, if you are running a stable NetBSD release in a production environment, you are encouraged to perform this procedure regularly in order to incorporate any security fixes that have been applied to the branch since its release.

There are a variety of ways of achieving the goal of rebuilding NetBSD from source, and this chapter will guide you through the variety of options that are available. The chapter starts by showing first what the manual procedure looks like, and proceeds to describe some of automation tools that simplify the process.

Note

Please remember to check src/UPDATING for the latest changes and special instructions that may be involved in upgrading the system.

35.1. Manual build and update procedure

Most of the following steps can be done as ordinary user. Only the installation of a new kernel and the userland will require root privileges. Although /usr is choosen as the working directory in the following examples, the procedure can also take place in a user's home directory. Ordinary users have normally not the permissions to make changes in /usr, but this can be changed by root.

Having up-to-date sources is a prerequisite for the following steps. Section 32.4, “Fetching by CVS” informs about the ways to retrieve or update the sources for a release, stable or current branch (using CVS).

Please always refer to the output of build.sh -h and the files UPDATING and BUILDING for details - it's worth it, there are many options that can be set on the command line or in /etc/mk.conf

35.1.1. Building a new userland

The first step is to build the userland:

$ cd /usr/src
$ ./build.sh -O ../obj -T ../tools -U distribution

35.1.2. Building a new kernel

The next step will build the kernel:

$ cd /usr/src
$ ./build.sh -O ../obj -T ../tools -U kernel=<KERNEL>
     

35.1.3. Installing the kernel and userland

Installing the new kernel, rebooting (to ensure that the new kernel works) and installing the new userland are the final steps of the updating procedure:

$ cd /usr/src
$ su
# mv /netbsd /netbsd.old
# mv /usr/obj/sys/arch/<ARCH>/compile/<KERNEL>/netbsd /
# shutdown -r now
 ...
$ cd /usr/src
$ su
# ./build.sh -O ../obj -T ../tools -U install=/

If the new kernel netbsd does not boot successfully, you can fall back on booting the netbsd.old kernel.

35.1.4. Updating the system configuration files

Updating your system's configuration files is done in two steps. First, postinstall(8) is used to check and fix things that can be easily automated. Afterwards, etcupdate(8) is used to merge the remaining configuration file changes.

# /usr/sbin/postinstall -s /usr/src check
# /usr/sbin/postinstall -s /usr/src fix
# /usr/sbin/etcupdate -s /usr/src

Optionally reboot to ensure all running services are using the new binaries:

# shutdown -r now

35.1.5. Summary

  1. From the root of the source tree:

    $ cd /usr/src
  2. Build the userland:

    $ ./build.sh -O ../obj -T ../tools -U -u distribution
  3. Build the kernel:

    $ ./build.sh -O ../obj -T ../tools -U -u kernel=GENERIC
  4. Install the kernel:

    $ cd ../obj/sys/arch/<ARCH>/compile/GENERIC
    $ su
    # mv /netbsd /netbsd.old
    # cp netbsd /netbsd
  5. Reboot into the new kernel:

    # shutdown -r now
  6. Install the new userland:

    $ cd /usr/src
    $ su
    # ./build.sh -O ../obj -T ../tools -U install=/
  7. Update the system and configuration files;:

    #  /usr/sbin/etcupdate -s /usr/src

Note

In the procedure above, the -u option indicates an update process, and that a make clean operation should not be run before starting the build. This is useful when doing an update from a previous build and/or a fresh build. The -U option allows the entire build by a non-root user followed with an install by root.

35.2. Using sysinst

It is also possible to use sysinst to install a freshly built system. The steps are as follows:

  1. Build a complete release:

    $ ./build.sh -O ../obj -T ../tools -U -u -x release

  2. The resulting install sets will be in the /usr/obj/releasedir/ directory.

  3. Copy the install kernel to the root directory of your NetBSD system, reboot from it, and upgrade with sysinst (see Chapter 4, Upgrading NetBSD).

35.3. Using sysbuild and sysupgrade

The sysbuild and sysupgrade tools (currently available in pkgsrc/sysutils/sysbuild and pkgsrc/sysutils/sysupgrade respectively) automate the full process of rebuilding NetBSD from sources (including the retrieval of the sources from a CVS repository) and installing the results with minimal effort.

Both of these tools have configuration files to determine how to build a release and how to install it. Among other things, these specify the CVS repository to use, what architecture to build for, where to place the build files and what steps to perform during an upgrade. The files can be found in /usr/pkg/etc/sysbuild/default.conf and /usr/pkg/etc/sysupgrade.conf. The default configuration of both tools should let you get started with minimal effort.

In their simplest form, you can do a full NetBSD build and upgrade your system to it by running these commands:

# sysbuild build
# sysupgrade auto ~/sysbuild/release/$(uname -m)
  

And that's all that it takes. These invocations will do the following:

  1. Download the source trees from CVS into /usr/src and /usr/xsrc. The latter is only fetched if your system has X11. And, if you already have the sources in your system, this will only update them to the newest version.

  2. Build a new release into ~/sysbuild/<machine>/. This per-machine directory will include subdirectories like obj, destdir, etc. The build results will be left in ~/sysbuild/release/<machine>/.

  3. Install a new kernel and unpack the new sets using the just-built release files.

  4. Run both etcupdate and postinstall to aid you in merging new configuration changes into your system.

For more details, please see the included sysbuild(1) and sysupgrade(8) manual pages, as well as the comments in the referenced configuration files.

35.3.1. Tweak: Building as non-root

The commands above depict the most basic and simple invocation of the tools using the default configuration files. One drawback is that you require root access during the build of the source tree so that sysbuild can upgrade the source trees under /usr/src and /usr/xsrc. It is recommended that you avoid building as root once you are familiar with the procedure, and this section show what is needed to do so with sysbuild.

In order to build as non-root, you can either choose to store your source trees out of /usr (easiest) or give permissions to your user to modify the trees under /usr (good if you want to share the source tree with more than one user).

If you want to store the source trees under your home directory, which is convenient for development purposes, simply edit /usr/pkg/etc/sysbuild.conf and add these settings:

SRCDIR="${HOME}/sysbuild/src"
[ ! -f /etc/mtree/set.xbase ] || XSRCDIR="${HOME}/sysbuild/xsrc"
    

Once this is done, the "sysbuild build" invocation show above should just work under your unprivileged user. The upgrade procedure then becomes:

$ sysbuild build
... become root ...
# sysupgrade auto ~/sysbuild/release/$(uname -m)
  

The other alternative, in case you want to maintain your source trees in the locations described by hier(7), is to do the following as root:

# mkdir -p /usr/src /usr/xsrc
# chown -R <your-user>:wsrc /usr/src /usr/xsrc
... and optionally add <your-user> to wsrc in /etc/group ...
    

After this, the default configuration file of sysbuild will let you place the files in these locations and let you do unprivileged builds.

35.3.2. Tweak: Setting up nightly builds

The pkgsrc/sysutils/sysbuild-user package can be used to configure and maintain an unprivileged system user to perform periodic (e.g. nightly) builds from source. This can come in very handy to closely track NetBSD-current.

The installed user is appropriately named sysbuild, and is configured by default to run a full system build overnight. The results are left in /home/sysbuild/release/<machine>/, which is the convenient default of sysupgrade's release directory. Any build failures will be reported to you by email.

The behavior of sysbuild for this unprivileged user is configured in /home/sysbuild/default.conf.

You can interact with sysbuild under this unprivileged user by running commands of the form:

# su - sysbuild /usr/pkg/bin/sysbuild ...
    

35.4. More details about the updating of configuration and startup files

etcupdate is a script to help users compare, merge and install new configuration and startup files (files found in the etc.tgz distribution set) in /dev, /etc and /root after performing an operating system upgrade. The upgrade of the operating system could have been performed either by compiling sources or by extracting the distribution binaries.

35.4.1. Using etcupdate with source files

In case where the sources are in /usr/src the following command should be enough:

# etcupdate

But what if your NetBSD sources are in an alternative location, such as in /home/jdoe/netbsd/src? Don't worry, tell etcupdate the location of your source tree with -s srcdir and it will work just fine:

# etcupdate -s /home/jdoe/netbsd/src

35.4.2. Using etcupdate with binary distribution sets

Sometimes it's not convenient to have the sources around but you still want to update the configuration and startup files. The solution is to feed etc.tgz (or xetc.tgz) to etcupdate via the -s tgzfile switch.

# etcupdate -s /some/where/etc.tgz

35.4.3. Using etcmanage instead of etcupdate

The etcmanage perl script (available from pkgsrc/sysutils/etcmanage or as binary package) is an alternative to etcupdate. It should be used in the following way, in combination with postinstall(8):

# /usr/pkg/bin/etcmanage
# /usr/sbin/postinstall