netboot: Scripts for building a PXE-style netboot image for KVM

A set of scripts and a short documentation describing how to build
a network boot image simulating a PXELINUX-style boot process.

Starting with QEMU 2.10 it is possible to boot a KVM guest over a
network interface using DHCP/BOOTP. The boot process is triggered by
the network boot firmware that is part of QEMU and follows the usual
network boot pattern: a DHCP request is issued by the client and
answered by a DHCP/BOOTP server. The DHCP reply will contain a
TFPT server identification and a bootfile name. The client will
retrieve the bootfile from the TFTP server, load it into memory
and IPL it.

A very common way of setting up a boot server has been defined
by PXELINUX, an open source implementation of PXE. With PXELINUX
the bootfile is a small network boot loader that will retrieve
a potentially client-specific configuration file containing
further instructions for the final boot process (kernel, ramdisk,...).

The set of sample scripts contained in the netboot directory provide
directions for a Linux distributor or a boot server administrator
on how to build a network boot image usable for a simplified
PXELINUX-style network boot setup for s390.

Note that the sample scripts are implementing only a subset of PXELINUX
functionality, specifically the config file parsing. In order to
get full functionality, a more specialized boot loader program
like petitboot or pxe-kexec must be used in the ramdisk.

Further, a sample Dockerfile is provided along with instructions on how
to build the network boot image in a Docker container.

Signed-off-by: Viktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: Michael Holzheu <holzheu@linux.vnet.ibm.com>
This commit is contained in:
Viktor Mihajlovski
2017-08-14 09:42:51 +02:00
committed by Michael Holzheu
parent d029030308
commit f3f16a7ca4
8 changed files with 760 additions and 1 deletions

21
netboot/Dockerfile Normal file
View File

@@ -0,0 +1,21 @@
#
# Sample Dockerfile to build PXE-style boot image for KVM on s390
#
FROM s390x/ubuntu:16.04
RUN apt-get update && apt-get install -y \
linux-image-4.4.0-78-generic \
make \
wget \
bzip2 \
linux-headers-4.4.0-78-generic \
gcc \
kexec-tools \
file
RUN mkdir /netboot
COPY . /netboot
RUN cd /netboot && make -f Makefile.pxelinux.0 KERNEL_VERSION=4.4.0-78-generic

16
netboot/Makefile Normal file
View File

@@ -0,0 +1,16 @@
# Install the netboot image build scripts as samples
include ../common.mak
NETBOOT_SAMPLEDIR=$(TOOLS_DATADIR)/netboot
all:
install: all
$(INSTALL) -d -m 755 $(DESTDIR)$(NETBOOT_SAMPLEDIR)
$(INSTALL) -m 755 mk-s390image mk-pxelinux-ramfs \
$(DESTDIR)$(NETBOOT_SAMPLEDIR)
$(INSTALL) -m 644 Dockerfile Makefile.pxelinux.0 README.md \
$(DESTDIR)$(NETBOOT_SAMPLEDIR)
.PHONY: all install clean

View File

@@ -0,0 +1,46 @@
# Sample Makefile to produce a pxe-style netboot image
# matching the currently active kernel.
# If the image is to be build for a different kernel version
# the KERNEL_VERSION variable must be overridden
# To use a local busybox installation, change BBINSTALL
# to point to the local busybox install path
ifeq ("$(KERNEL_VERSION)","")
KERNEL_VERSION=$(shell uname -r)
endif
export KERNEL_VERSION
filetype=$(shell file $1 | grep "Linux S390")
s390_check=$(if $(call filetype,$1), $1)
prefixes=image vmlinux vmlinuz
kernel_images=$(foreach prefix, $(prefixes), \
$(call s390_check,/boot/$(prefix)-$(KERNEL_VERSION)))
KERNEL_IMAGE=$(firstword $(kernel_images))
ifeq ($(KERNEL_IMAGE),)
$(error Could not find a kernel image under /boot)
endif
BUSYBOX=busybox-1.27.1
BBINSTALL=$(BUSYBOX)/_install
all: $(KERNEL_IMAGE) pxelinux.initramfs
/bin/bash mk-s390image $(KERNEL_IMAGE) pxelinux.0 -r pxelinux.initramfs
pxelinux.initramfs: $(BBINSTALL)
/bin/bash mk-pxelinux-ramfs -b $< -k $(KERNEL_VERSION) $@
$(BUSYBOX)/_install:
wget https://busybox.net/downloads/$(BUSYBOX).tar.bz2
tar xjf $(BUSYBOX).tar.bz2
make -C $(BUSYBOX) defconfig
make -C $(BUSYBOX) install
install:
clean:
$(RM) pxelinux.0 pxelinux.initramfs $(BUSYBOX).tar.bz2
$(RM) -r $(BUSYBOX)
.PHONY: all install clean

99
netboot/README.md Normal file
View File

@@ -0,0 +1,99 @@
# How to build a PXELINUX-style network boot image for KVM
## Synopsis
To build a PXELINUX-style netboot image usable for KVM a s390 Linux system
with access to the internet is required.
Running the following command will generate the netboot image pxlinux.0:
` $ make -f Makefile.pxelinux.0`
Alternatively you can use docker to build the image:
```
$ docker build -t pxelinux0 .
$ docker run --rm -v $(pwd):/out pxelinux0 cp /netboot/pxelinux.0 /out
$ docker rmi pxelinux0
```
The resulting file pxelinux.0 must be copied to the system acting as
DHCP/BOOTP server for the KVM installation.
## Full Description
Starting with QEMU 2.10 it is possible to boot s390 virtual machines over a
network interface using DHCP. As usual for DHCP/BOOTP a single bootable image
is copied from the boot server, loaded into memory and booted. In order to
boot a Linux Operating System, it is typically necessary to load a kernel
together with an initial ramdisk (initramfs) and optionally specify some kernel
command line parameters.
Alternatively, on s390 it is possible to load a single file consisting of
the kernel image followed by an initial ramdisk. Such single boot images can
be provided by a Linux distributor, e.g. on the installation media.
Single boot images can also easily be built from pre-existing kernel/initramfs
pairs by concatenating these files. In order to allow the kernel to find the
ramdisk, it is necessary to update the 8 bytes at location 0x10408 with the
offset value of the ramdisk in the new binary, and the 8 bytes at location
0x10410 with the size of the ramdisk. Both values need to be updated in binary,
big endian format.
Since PXELINUX, the PXE boot implementation provided by the Syslinux project,
has introduced a popular way to set up network boot servers for Linux, it
is desirable that s390 network boot setups can be done in a similar way.
A boot image simulating a PXELINUX-like boot for s390 can be easily
constructed by combining a Linux kernel with a small fit-to-purpose initial
ramdisk as described above. For practical purposes, using the host kernel is
a reasonable way for this kind of approach. If possible, the initial ramdisk
should be independent of the host, which is not always possible, as the kernel
might require modules for e.g. virtio network and block devices.
### Example: Building a PXELINUX-style boot image
The approach described below consists of bundling some shell scripts, busybox
and the kexec binary bundled into the initial ramdisk.
The init process can be a simple shell script that will mount a few essential
file systems, like /dev, /proc, and /sys, start a DHCP client (e.g. busybox's
udchpc) and then invoke another script to perform the network boot.
udchpc will invoke the script /usr/share/udhcpc/default.script in response
to DHCP server messages to perform configuration actions.
The sample default.script delivered with busybox can be used for that purpose,
but needs to be extended to evaluate the bootp specific DHCP options (most
important the tftp server address) and store them for use by the boot script.
The boot script itself has to retrieve the PXELINUX configuration from the
tftp server according to the rules described [here][1] then retrieve the
remote kernel and initial ramdisk and finally use kexec to boot the network
kernel.
In essence, the following steps are performed to produce the initial
ramdisk:
1. Create a skeleton initramfs directory structure
2. Create the init script, the boot script and the DHCP default script
3. Copy kexec and it's dependencies from the host into the initramfs
4. Copy virtio network and block modules of the host's active kernel into
the initramfs
5. Copy the busybox binaries into the initramfs.
6. Copy the DHCP configuration and PXE boot scripts to the initramfs
7. Build the ramdisk (in compressed CPIO format)
8. Concatenate the kernel image and the initial ramdisk, and adjust the
ramdisk offset as described above.
Steps 1 to 7 are performed by the sample script mk-pxelinux-ramfs, while step
8 is done with the help of mk-s390image.
The binary resulting from the procedure described above can now be deployed
to a DHCP/BOOTP server. This server should also act as a TFTP server for the
PXELINUX configuration and binary files needed to complete the network boot.
Alternatively, it is possible to use programs like [petitboot][2] or
[pxe-kexec][3] in the initial ramdisk, as these provide more sophisticated
and robust processing of PXELINUX-style configurations.
[1]: http://www.syslinux.org/wiki/index.php?title=PXELINUX
[2]: https://github.com/open-power/petitboot
[3]: https://sourceforge.net/projects/pxe-kexec.berlios/

402
netboot/mk-pxelinux-ramfs Executable file
View File

@@ -0,0 +1,402 @@
#!/bin/bash
#
# netboot - PXE-style boot for KVM on s390
#
# Sample script to build an initramfs image suitable for performing a
# PXELINUX-style boot for KVM guests from a DHCP/BOOTP Server.
# Has to be executed on the KVM host where it will be deployed and needs
# to be re-run after kernel updates on the host, unless the virtio
# drivers are statically built into the host kernel.
#
# The script requires a busybox install tree, e.g. resulting from a build
# from source, after make defconfig && make install
#
# To keep things simple, we don't include udev but use devtmpfs
# which means the host kernel must have been built with CONFIG_DEVTMPFS=y
#
# Sample invocation:
#
# ./mk-pxelinux-ramfs -b /downloads/busyboxdir pxelinux.initramfs
#
# Copyright IBM Corp. 2017
#
# s390-tools is free software; you can redistribute it and/or modify
# it under the terms of the MIT license. See LICENSE for details.
# Variables
cmd=$(basename $0)
busyboxdir=
builddir=
initramfs=
success=no
# Cleanup on exit
cleanup()
{
if [ -n $builddir ]
then
rm -rf $builddir
fi
}
trap cleanup EXIT
# Usage
usage()
{
cat <<-EOF
Usage: $cmd -b BUSYBOX_DIR [-k KERNEL_VERSION] INITRAMFS_FILE
Build a PXELINUX style boot initramfs INITRAMFS_FILE using a busybox installed
in BUSYBOX_DIR and kernel modules from the currently running kernel or from
the kernel version specified with the '-k KERNEL_VERSION' option.
OPTIONS
-b Search installed busybox in directory BUSYBOX_DIR
-k Use KERNEL_VERSION instead of currently running kernel
-h Print this help, then exit
EOF
}
# Get shared objects for binary
sharedobjs()
{
ldd $1 | sed -e 's?[^/]*??' -e 's/(.*)//'
}
# Check args
args=$(getopt b:k:h $*)
if [ $? = 0 ]
then
set -- $args
while [ -n $1 ]
do
case $1 in
-b) busyboxdir=$2; shift 2;;
-k) kernelversion=$2; shift 2;;
-h) usage; exit 0;;
--) shift; break;;
*) echo "$cmd: Unexpected argument $1, exiting..." >&2; exit 1;;
esac
done
fi
if [ $# != 1 -o "$busyboxdir" = "" ]
then
usage >&2
exit 1
fi
# Full output file path
initramfs=$(readlink -m $(dirname $1))/$(basename $1)
# Exit on error
set -e
# Module locations
if [ -n $kernelversion ]; then
moddir=/lib/modules/$kernelversion
else
moddir=/lib/modules/$(uname -r)
fi
netdir=$moddir/kernel/drivers/net
blkdir=$moddir/kernel/drivers/block
# Setup build directory
builddir=$(mktemp -d)
echo "$cmd: Building in $builddir"
ramfsdirs="/bin /dev /etc /lib64 /lib /mnt /proc /run /sbin /sys /tmp /usr /var"
for d in $ramfsdirs
do
mkdir -p $builddir/$d
done
# Kexec
echo "$cmd: Copying kexec"
# Install both binary and required shared libraries
OLDPATH=$PATH
PATH=$OLDPATH:/sbin:/usr/sbin
kexec_bin=$(which kexec)
kexec_sos=$(sharedobjs $kexec_bin)
PATH=$OLDPATH
cp $kexec_bin $builddir/sbin
for so in $kexec_sos
do
mkdir -p $builddir/$(dirname $so)
cp $so $builddir/$(dirname $so)
done
# virtio module(s), if present
echo "$cmd: Copying virtio modules"
mkdir -p $builddir/$netdir
mkdir -p $builddir/$blkdir
set +e
cp $netdir/virtio_net.ko $builddir/$netdir 2> /dev/null
cp $blkdir/virtio_blk.ko $builddir/$blkdir 2> /dev/null
set -e
# Busybox (+ dependencies)
echo "$cmd: Copying busybox files"
cp -a $busyboxdir/* $builddir
busybox_sos=$(sharedobjs $busyboxdir/bin/busybox)
for so in $busybox_sos
do
mkdir -p $builddir/$(dirname $so)
cp $so $builddir/$(dirname $so)
done
# ad_packet module(s), if present
echo "$cmd: Copying af_packet modules"
packetdir=$moddir/kernel/net/packet
mkdir -p $builddir/$packetdir
set +e
cp $packetdir/* $builddir/$packetdir 2> /dev/null
set -e
# Init script
echo "$cmd: Making init script"
# --- begin init script
cat <<'EOF' > $builddir/init
#!/bin/sh
/bin/mount -t devtmpfs none /dev
/bin/mount -t proc none /proc
/bin/mount -t sysfs none /sys
/bin/mount -t tmpfs none /run
/sbin/modprobe virtio_net
/sbin/udhcpc -O pxeconffile -O pxepathprefix &
while true
do
/sbin/pxeboot.script
/bin/sleep 1
done
EOF
# --- end init script
chmod +x $builddir/init
# udhcpc script
echo "$cmd: Making DHCP script"
mkdir -p $builddir/usr/share/udhcpc
# -- begin dhcp script
cat <<'EOF' > $builddir/usr/share/udhcpc/default.script
#!/bin/sh
# Setup name resolution and PXE boot configuration
# called by udhcpc
RESOLVCONF="/etc/resolv.conf"
PXECONF="/etc/pxe.conf"
ccidr()
{
# clumsy netmask to cidr transformation
# with minimal sanity checking
OLDIFS=$IFS
IFS=.
c=0
n=4
for i in $1
do
n=$(/usr/bin/expr $n - 1)
case $i in
255) c=$(/usr/bin/expr $c + 8);;
254) c=$(/usr/bin/expr $c + 7); break;;
252) c=$(/usr/bin/expr $c + 6); break;;
248) c=$(/usr/bin/expr $c + 5); break;;
240) c=$(/usr/bin/expr $c + 4); break;;
224) c=$(/usr/bin/expr $c + 3); break;;
192) c=$(/usr/bin/expr $c + 2); break;;
128) c=$(/usr/bin/expr $c + 1); break;;
0) break;;
*) c=0; break;;
esac
if [ $n = 0 ]
then
break
fi
done
IFS=$OLDIFS
echo $c
}
echo "DHCP response $1: "
case "$1" in
deconfig)
echo " interface: $interface"
/sbin/ip route flush table all
/sbin/ip addr flush $interface
/sbin/ip link set $interface up
/bin/rm -f $PXECONF $RESOLVCONF
;;
renew|bound)
echo " interface: $interface $ip $subnet"
echo " router: $router"
echo " domain: $domain $dns"
echo " tftp: $siaddr"
# flush routes
/sbin/ip route flush table all
# setup if link
/sbin/ip addr flush $interface
/sbin/ip link set $interface up
# setup if addr
if [ -n "$subnet" ]
then
maskedip="$ip"/$(ccidr $subnet)
else
maskedip="ip"
fi
/sbin/ip addr add $maskedip broadcast $broadcast dev $interface
# setup default routes
if [ -n "$router" ]
then
/sbin/ip route add default via $router
fi
# setup resolv.conf
if [ -n "$domain" ]
then
echo "search $domain" > $RESOLVCONF
for i in $dns
do
echo " nameserver $i" >> $RESOLVCONF
done
fi
# pxe control
if [ -n "$siaddr" ]
then
echo "siaddr=$siaddr" > $PXECONF
echo "interface=$interface" >> $PXECONF
echo "ip=$ip" >> $PXECONF
fi
;;
nak)
;;
*)
exit 1
;;
esac
exit 0
EOF
# -- end dhcp script
chmod +x $builddir/usr/share/udhcpc/default.script
# pxeboot script
echo "$cmd: Making PXE boot script"
# -- begin pxeboot script
cat <<'EOF' > $builddir/sbin/pxeboot.script
#!/bin/sh
# Perform a PXE style boot using kexec
# Supports only super-simple config files
set -e
# Source the DHCP-generated TFTP info
# Currently we are just looking for siaddr
PXE_CONF=/etc/pxe.conf
if [ -f $PXE_CONF ]
then
. $PXE_CONF
else
echo waiting for PXE configuration
exit
fi
# Retrieve the config (default only for now)
CONFIGS=""
if [ -n "$siaddr" ];
then
# Enable UUID based config on s390
if [ $(/bin/uname -m) = "s390x" ]
then
set +e
uuid=$(/bin/grep UUID /proc/sysinfo | tail -1 | tr -d ' ' | cut -d ':' -f 2) 2> /dev/null
set -e
else
# not caring for other arches right now
uuid=""
fi
CONFIGS="$CONFIGS $uuid"
# Enable MAC based config
ifaddr=$(/bin/cat /sys/class/net/$interface/address | tr ':' '-')
CONFIGS="$CONFIGS 01-$ifaddr"
# Enable IP based config
iphex=$(printf %02X $(echo $ip | tr '.' ' '))
for i in 8 7 6 5 4 3 2 1
do
CONFIGS="$CONFIGS $(echo $iphex | cut -c 1-$i)"
done
# Finally enable default config
CONFIGS="$CONFIGS default"
set +e
for c in $CONFIGS
do
if /usr/bin/tftp -g -l /tmp/config -r pxelinux.cfg/$c $siaddr
then
break
fi
done
fi
if [ ! -f /tmp/config ]
then
echo no config found
exit
fi
# Simple config file parsing, only one entry allowed
kernel=$(/bin/grep "^[[:space:]]*kernel" /tmp/config | sed "s/^[[:space:]]*kernel[[:space:]]*//")
initrd=$(/bin/grep "^[[:space:]]*initrd" /tmp/config | sed "s/^[[:space:]]*initrd[[:space:]]*//")
append=$(/bin/grep "^[[:space:]]*append" /tmp/config | sed "s/^[[:space:]]*append[[:space:]]*//")
if [ -z "$kernel" ]
then
echo no kernel statement found in config
exit
else
/usr/bin/tftp -g -l /tmp/kernel -r $kernel $siaddr
fi
if [ -n "$initrd" ]
then
/usr/bin/tftp -g -l /tmp/initrd -r $initrd $siaddr
INITRD="--initrd=/tmp/initrd"
else
INITRD=""
fi
if [ -n "$append" ]
then
APPEND="--append=$append"
else
APPEND=""
fi
kexec -l /tmp/kernel $INITRD $APPEND
kexec -e
EOF
# -- end pxeboot script
chmod +x $builddir/sbin/pxeboot.script
# The final initramfs
echo Building initramfs
cd $builddir
find . | cpio -o -Hnewc | gzip - > $initramfs
cd $OLDPWD

171
netboot/mk-s390image Executable file
View File

@@ -0,0 +1,171 @@
#!/bin/bash
#
# netboot - PXE-style boot for KVM on s390
#
# Sample script to build a single s390 boot image consisting of
# kernel, an initial ramdisk and kernel parameters from
# individual components. Note that bash is required to run this script!
#
# Sample invocation:
#
# ./mk-s390image /boot/image -r /boot/initrd image
#
# The resulting image can be used to build a bootable
# ISO or as firmware image for KVM.
#
# Copyright IBM Corp. 2017
#
# s390-tools is free software; you can redistribute it and/or modify
# it under the terms of the MIT license. See LICENSE for details.
# Offsets
OFFS_INITRD_START_BYTES=66568
OFFS_INITRD_SIZE_BYTES=66576
OFFS_COMMANDLINE_BYTES=66688
MAX_PARMFILE_SIZE=896
# Variables
cmd=$(basename $0)
kernel=
ramdisk=
parmfile=
image=
binval=
success=no
# Cleanup on exit
cleanup()
{
if [ -n "$binval" ]
then
rm -f $binval
fi
if [ -n "$image" -a $success = no ]
then
rm $image
fi
}
trap cleanup EXIT
# Usage
usage()
{
cat <<-EOF
Usage: $cmd KERNEL BOOT_IMAGE [-r RAMDISK] [-p PARMFILE]
Build an s390 image BOOT_IMAGE suitable for CD/tape/network boot or as a
KVM firmware image using a stripped Linux kernel file KERNEL.
OPTIONS
-p Use PARMFILE with kernel parameters in the image
-r Include RAMDISK in the image
-h Print this help, then exit
EOF
}
# Convert decimal number to big endian doubleword
dec2be64()
{
local num=$1
local b
local i
for i in $(seq 1 8)
do
b="\\x$(printf '%x' $(expr $num % 256))$b"
num=$(expr $num / 256) || true
done
printf $b
}
# Do the image build
dobuild()
{
local i
local kernel_size
local ramdisk_size
local ramdisk_offset
local parmfile_size
# check whether all specified files exist
for i in $kernel $ramdisk $parmfile
do
if [ ! -f $i ]
then
echo "$cmd: File $i not found" >&2
return 1
fi
done
if ! file -b $(readlink -f $kernel) | grep "Linux S390" > /dev/null
then
echo "$cmd: Unrecognized file format for $kernel" >&2
return 1
fi
# from now on we SHOULD only fail on disk shortage
# or file permissions, let the shell handle that
set -e
# copy over kernel padded with zeroes to page boundary
dd if=$kernel of=$image bs=4096 conv=sync status=none
# append ramdisk if specified
if [ "$ramdisk" != "" ]
then
ramdisk_size=$(du -b $ramdisk | cut -f1)
kernel_size=$(du -b $kernel | cut -f1)
ramdisk_offset=$(du -b $image | cut -f1)
cat $ramdisk >> $image
binval=$(mktemp)
dec2be64 $ramdisk_offset > $binval
dd seek=$OFFS_INITRD_START_BYTES if=$binval of=$image bs=1 \
count=8 conv=notrunc status=none
dec2be64 $ramdisk_size > $binval
dd seek=$OFFS_INITRD_SIZE_BYTES if=$binval of=$image bs=1 \
count=8 conv=notrunc status=none
fi
# set cmdline
if [ "$parmfile" != "" ]
then
parmfile_size=$(du -b $parmfile | cut -f1)
if [ $parmfile_size -le $MAX_PARMFILE_SIZE ]
then
dd seek=$OFFS_COMMANDLINE_BYTES bs=1 if=$parmfile \
of=$image conv=notrunc status=none
else
echo "$cmd: Size $parmfile_size of $parmfile exceeds command line limit of $MAX_PARMFILE_SIZE" >&2
return 1
fi
fi
# we've done it
success=yes
}
# check args and build
args=$(getopt "r:p:h" $*)
if [ $? = 0 ]
then
set -- $args
while [ $1 != "" ]
do
case $1 in
-r) ramdisk=$2; shift 2;;
-p) parmfile=$2; shift 2;;
-h) usage; exit 0;;
--) shift; break;;
*) echo "$cmd: Unexpected argument $1, exiting..." >&2; exit 1;;
esac
done
fi
if [ $# = 2 ]
then
kernel=$1
image=$2
dobuild
exit 0
fi
# something wasn't right
usage >&2
exit 1