Documentation reorg for v4.6.0
Signed-off-by: Jonathon Anderson <janderson@ciq.com>
This commit is contained in:
Jonathon Anderson
393
userdocs/server/bootloaders.rst
Normal file
393
userdocs/server/bootloaders.rst
Normal file
@@ -0,0 +1,393 @@
|
||||
===========
|
||||
Bootloaders
|
||||
===========
|
||||
|
||||
Warewulf uses iPXE as its default network bootloader. As a tech preview, support
|
||||
for GRUB is also available, which adds support for secure boot.
|
||||
|
||||
Also as a tech preview, Warewulf may also use iPXE or GRUB to boot a dracut
|
||||
initramfs as an initial stage before loading the image. This is called a
|
||||
two-stage boot.
|
||||
|
||||
Booting with iPXE
|
||||
=================
|
||||
|
||||
The ``/etc/warewulf/ipxe/`` directory contains *text/templates* that are used by
|
||||
the Warewulf configuration process to configure the ``ipxe`` service.
|
||||
|
||||
.. graphviz::
|
||||
|
||||
digraph G{
|
||||
node [shape=box];
|
||||
compound=true;
|
||||
edge [label2node=true]
|
||||
bios [shape=record label="{BIOS | boots from DHCP/next-server via TFTP}"]
|
||||
|
||||
subgraph cluster0 {
|
||||
label="iPXE boot"
|
||||
iPXE;
|
||||
ipxe_cfg [shape=record label="{ipxe.cfg|generated for each node}"];
|
||||
iPXE -> ipxe_cfg [label="http"];
|
||||
}
|
||||
|
||||
bios->iPXE [lhead=cluster0,label="iPXE.efi"];
|
||||
|
||||
kernel [shape=record label="{kernel|ramdisk (root fs)|wwinit overlay}|extracted from node image"];
|
||||
ipxe_cfg->kernel[ltail=cluster0,label="http"];
|
||||
}
|
||||
|
||||
Starting in v4.5.0, Warewulf no longer includes an iPXE binary. In stead, by
|
||||
default Warewulf uses the iPXE that comes with the host OS.
|
||||
|
||||
Unfortunately, we’ve encountered a few instances where bugs in the OS-provided
|
||||
iPXE that sometimes make booting a full OS image as an "initrd" unreliable.
|
||||
|
||||
:ref:`Building iPXE locally`, using a more recent "version" of the iPXE source
|
||||
code, can alleviate some of these issues.
|
||||
|
||||
Another alternative is :ref:`booting with dracut`, which uses the Linux kernel
|
||||
to load the full OS image, avoiding the issue entirely.
|
||||
|
||||
.. _Building iPXE locally:
|
||||
|
||||
Building iPXE locally
|
||||
---------------------
|
||||
|
||||
By default (as of v4.5.0) Warewulf packages use iPXE from the host operating
|
||||
system rather than bundling iPXE binaries with Warewulf. However, sometimes the
|
||||
specific build included in the host OS has bugs or missing features, and a local
|
||||
build of iPXE is necessary.
|
||||
|
||||
The Warewulf project provides a `build-ipxe.sh`_ script to simplify the process
|
||||
of building iPXE locally.
|
||||
|
||||
.. _build-ipxe.sh: https://github.com/warewulf/warewulf/blob/main/scripts/build-ipxe.sh
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# curl -LO https://raw.githubusercontent.com/warewulf/warewulf/main/scripts/build-ipxe.sh
|
||||
# bash build-ipxe.sh -h
|
||||
Usage: build-ipxe.sh
|
||||
[-h] (help)
|
||||
TARGETS: bin-x86_64-pcbios/undionly.kpxe bin-x86_64-efi/snponly.efi bin-arm64-efi/snponly.efi
|
||||
IPXE_BRANCH: master
|
||||
DESTDIR: /usr/local/share/ipxe
|
||||
|
||||
Running build-ipxe.sh
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The script, by default, builds iPXE for x86_64 BIOS, x86_64 EFI, and arm64 EFI
|
||||
from the master branch on the iPXE project GitHub and stores the resultant
|
||||
builds in ``/usr/local/share/ipxe/``. (These parameters can be adjusted by
|
||||
setting ``TARGETS``, ``IPXE_BRANCH``, and ``DESTDIR`` environment variables,
|
||||
with the current values shown in the ``-h`` output for reference.)
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# mkdir -p /usr/local/share/ipxe
|
||||
# bash build-ipxe.sh
|
||||
[...]
|
||||
# ls -1 /usr/local/share/ipxe/
|
||||
bin-arm64-efi-snponly.efi
|
||||
bin-x86_64-efi-snponly.efi
|
||||
bin-x86_64-pcbios-undionly.kpxe
|
||||
|
||||
.. note::
|
||||
|
||||
Building for aarch64 requires the package ``gcc-aarch64-linux-gnu``.
|
||||
|
||||
Build options
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
By default, ``build-ipxe.sh`` enables support for `ZLIB`_ and `GZIP`_ images, as
|
||||
well as commands for managing `VLANs`_ and the `framebuffer console`_. The
|
||||
x86_64 build also enables support for the `serial console`_.
|
||||
|
||||
.. _ZLIB: https://ipxe.org/buildcfg/image_zlib
|
||||
|
||||
.. _GZIP: https://ipxe.org/buildcfg/image_gzip
|
||||
|
||||
.. _VLANs: https://ipxe.org/buildcfg/vlan_cmd
|
||||
|
||||
.. _framebuffer console: https://ipxe.org/buildcfg/console_framebuffer
|
||||
|
||||
.. _serial console: https://ipxe.org/buildcfg/console_serial
|
||||
|
||||
Additional `build options`_ can be configured by editing the ``build-ipxe.sh`` script.
|
||||
For example, the x86_64 build is configured in the ``configure_x86_64`` function.
|
||||
|
||||
.. _build options: https://ipxe.org/buildcfg
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
function configure_x86_64 {
|
||||
sed -i.bak \
|
||||
-e 's,//\(#define.*CONSOLE_SERIAL.*\),\1,' \
|
||||
-e 's,//\(#define.*CONSOLE_FRAMEBUFFER.*\),\1,' \
|
||||
config/console.h
|
||||
sed -i.bak \
|
||||
-e 's,//\(#define.*IMAGE_ZLIB.*\),\1,' \
|
||||
-e 's,//\(#define.*IMAGE_GZIP.*\),\1,' \
|
||||
-e 's,//\(#define.*VLAN_CMD.*\),\1,' \
|
||||
config/general.h
|
||||
}
|
||||
|
||||
For example, the ``imgextract`` command can be `explicitly enabled`_.
|
||||
|
||||
.. _explicitly enabled: https://ipxe.org/buildcfg/image_archive_cmd
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
function configure_x86_64 {
|
||||
sed -i.bak \
|
||||
-e 's,//\(#define.*CONSOLE_SERIAL.*\),\1,' \
|
||||
-e 's,//\(#define.*CONSOLE_FRAMEBUFFER.*\),\1,' \
|
||||
config/console.h
|
||||
sed -i.bak \
|
||||
-e 's,//\(#define.*IMAGE_ZLIB.*\),\1,' \
|
||||
-e 's,//\(#define.*IMAGE_GZIP.*\),\1,' \
|
||||
-e 's,//\(#define.*VLAN_CMD.*\),\1,' \
|
||||
-e 's,//\(#define.*IMAGE_ARCHIVE_CMD.*\),\1,' \
|
||||
config/general.h
|
||||
}
|
||||
|
||||
.. note::
|
||||
|
||||
``IMG_ARCHIVE_CMD`` is already enabled by default in the iPXE master branch,
|
||||
but only takes effect when at least one archive image format is configured.
|
||||
This is the case in the default state of ``build-ipxe.sh``, which enables
|
||||
support for ZLIB and GZIP archive image formats.
|
||||
|
||||
Configuring Warewulf (≥ v4.5.0)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
In Warewulf v4.5.0, Warewulf can be configured to use these files using the
|
||||
``tftp.ipxe`` and ``paths.ipxesource`` configuration parameters in
|
||||
``warewulf.conf``.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
# warewulf.conf
|
||||
tftp:
|
||||
ipxe:
|
||||
"00:00": bin-x86_64-pcbios-undionly.kpxe
|
||||
"00:07": bin-x86_64-efi-snponly.efi
|
||||
"00:09": bin-x86_64-efi-snponly.efi
|
||||
"00:0B": bin-arm64-efi-snponly.efi
|
||||
paths:
|
||||
ipxesource: /usr/local/share/ipxe
|
||||
|
||||
Restart ``warewulfd`` following the change to ``warewulf.conf``. Then remove any
|
||||
previously-provisioned files from ``/var/lib/tftpboot/warewulf/`` and use
|
||||
``wwctl configure tftp`` and ``wwctl configure dhcp`` to re-provision the TFTP
|
||||
files and update the DHCP configuration.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# sudo systemctl restart warewulfd
|
||||
# rm /var/lib/tftpboot/warewulf/*
|
||||
# wwctl configure tftp
|
||||
Writing PXE files to: /var/lib/tftpboot/warewulf
|
||||
Enabling and restarting the TFTP services
|
||||
# wwctl configure dhcp
|
||||
Building overlay for wwctl1: host
|
||||
Enabling and restarting the DHCP services
|
||||
|
||||
Configuring Warewulf (< v4.5.0)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Prior to v4.5.0, Warewulf packages included bundled builds of iPXE and did not
|
||||
provide a mechanism for configuring which iPXE to use. To use a custom iPXE
|
||||
before v4.5.0, replace the bundled builds included with Warewulf. After that,
|
||||
remove any previously-provisioned files from ``/var/lib/tftpboot/warewulf/`` and
|
||||
use ``wwctl configure tftp`` to re-provision the TFTP files.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# cp /usr/local/share/ipxe/bin-arm64-efi-snponly.efi /usr/share/warewulf/ipxe/arm64.efi
|
||||
# cp /usr/local/share/ipxe/bin-x86_64-efi-snponly.efi /usr/share/warewulf/ipxe/x86_64.efi
|
||||
# cp /usr/local/share/ipxe/bin-x86_64-pcbios-undionly.kpxe /usr/share/warewulf/ipxe/x86_64.kpxe
|
||||
# rm /var/lib/tftpboot/warewulf/*
|
||||
# wwctl configure tftp
|
||||
Writing PXE files to: /var/lib/tftpboot/warewulf
|
||||
Enabling and restarting the TFTP services
|
||||
|
||||
Booting with GRUB
|
||||
=================
|
||||
|
||||
Support for GRUB as a network bootloader (replacing iPXE) is available in
|
||||
Warewulf as a technology preview.
|
||||
|
||||
.. graphviz::
|
||||
|
||||
digraph G{
|
||||
node [shape=box];
|
||||
compound=true;
|
||||
edge [label2node=true]
|
||||
bios [shape=record label="{BIOS | boots from DHCP/next-server via TFTP}"]
|
||||
|
||||
bios->shim [lhead=cluster1,label="shim.efi"];
|
||||
subgraph cluster1{
|
||||
label="Grub boot"
|
||||
shim[shape=record label="{shim.efi|from ww4 host}"];
|
||||
grub[shape=record label="{grubx64.efi | name hardcoded in shim.efi|from ww4 host}"]
|
||||
shim->grub[label="TFTP"];
|
||||
grubcfg[shape=record label="{grub.cfg|static under TFTP root}"];
|
||||
grub->grubcfg[label="TFTP"];
|
||||
}
|
||||
kernel [shape=record label="{kernel|ramdisk (root fs)|wwinit overlay}|extracted from node image"];
|
||||
grubcfg->kernel[ltail=cluster1,label="http"];
|
||||
}
|
||||
|
||||
Instead of the iPXE starter a combination of `shim and GRUB
|
||||
<https://www.suse.com/c/uefi-secure-boot-details/>`_ can be used with the
|
||||
advantage that secure boot can be used. That means that only the signed kernel
|
||||
of a distribution can be booted. This can be a huge security benefit for some
|
||||
scenarios.
|
||||
|
||||
In order to enable the grub boot method it has to be enabled in `warewulf.conf`.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
warewulf:
|
||||
grubboot: true
|
||||
|
||||
Nodes which are not known to Warewulf are booted with the shim/grub from the
|
||||
Warewulf server host.
|
||||
|
||||
Secure boot
|
||||
-----------
|
||||
|
||||
.. graphviz::
|
||||
|
||||
digraph foo {
|
||||
node [shape=box];
|
||||
subgraph boot {
|
||||
"EFI" [label="EFI",row=boot];
|
||||
"Shim" [label="Shim",row=boot];
|
||||
"Grub" [label="Grub",row=boot];
|
||||
"Kernel" [label="kernel",row=boot];
|
||||
EFI -> Shim[label="Check for Microsoft signature"];
|
||||
Shim -> Grub[label="Check for Distribution signature"];
|
||||
Grub->Kernel[label="Check for Distribution or MOK signature"];
|
||||
}
|
||||
}
|
||||
|
||||
If secure boot is enabled at every step a signature is checked and the boot
|
||||
process fails if this check fails. The shim typically only includes the key for
|
||||
a single operating system, which means that each distribution needs separate
|
||||
`shim` and `grub` executables. Warewulf extracts these binaries from the images.
|
||||
If the node is unknown to Warewulf or can't be identified during the TFTP boot
|
||||
phase, the shim/grub binaries of the host in which Warewulf is running are used.
|
||||
|
||||
Install shim and efi
|
||||
--------------------
|
||||
|
||||
`shim.efi` and `grub.efi` must be installed in the image for it to be
|
||||
booted by GRUB.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# wwctl image shell leap15.5
|
||||
[leap15.5] Warewulf> zypper install grub2 shim
|
||||
|
||||
# wwctl image shell rocky9
|
||||
[rocky9] Warewulf> dnf install shim-x64.x86_64 grub2-efi-x64.x86_64
|
||||
|
||||
These packages must also be installed on the Warewulf server host to enable
|
||||
node discovery using GRUB.
|
||||
|
||||
HTTP boot
|
||||
---------
|
||||
|
||||
Modern EFI systems have the possibility to directly boot per http. The flow
|
||||
diagram is the following:
|
||||
|
||||
.. graphviz::
|
||||
|
||||
digraph G{
|
||||
node [shape=box];
|
||||
efi [shape=record label="{EFI|boots from URI defined in filename}"];
|
||||
shim [shape=record label="{shim.efi|replaces shim.efi with grubx64.efi in URI|extracted from node image}"];
|
||||
grub [shape=record label="{grub.efi|checks for grub.cfg|extracted from node image}"]
|
||||
kernel [shape=record label="{kernel|ramdisk (root fs)|wwinit overlay}|extracted from node image"];
|
||||
efi->shim [label="http"];
|
||||
shim->grub [label="http"];
|
||||
grub->kernel [label="http"];
|
||||
}
|
||||
|
||||
Warewulf delivers the initial `shim.efi` and `grub.efi` via http as taken
|
||||
directly from the node's assigned image.
|
||||
|
||||
.. _booting with dracut:
|
||||
|
||||
Booting with dracut
|
||||
===================
|
||||
|
||||
Some systems, typically due to limitations in their BIOS or EFI firmware, are
|
||||
unable to load image of a certain size directly with a traditional bootloader,
|
||||
either iPXE or GRUB. As a workaround for such systems, Warewulf can be
|
||||
configured to load a dracut initramfs from the image and to use that initramfs
|
||||
to load the full image.
|
||||
|
||||
Warewulf provides a dracut module to configure the dracut initramfs to load the
|
||||
image. This module is available in the ``warewulf-dracut`` subpackage, which
|
||||
must be installed in the image.
|
||||
|
||||
With the ``warewulf-dracut`` package installed, you can build an initramfs
|
||||
inside the image.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
dnf -y install warewulf-dracut
|
||||
dracut --force --no-hostonly --add wwinit --regenerate-all
|
||||
|
||||
.. note::
|
||||
|
||||
In some systems, such as ``rockylinux:8``, it may be necessary to remove
|
||||
``/etc/machine-id`` for dracut to properly generate the initramfs in the
|
||||
location that Warewulf is expecting.
|
||||
|
||||
To direct iPXE to fetch the node's initramfs image and boot with dracut
|
||||
semantics, set an ``IPXEMenuEntry`` tag for the node.
|
||||
|
||||
.. note::
|
||||
|
||||
Warewulf configures iPXE with a template located at
|
||||
``/etc/warewulf/ipxe/default.ipxe``. Inspect the template to learn more about
|
||||
the dracut booting process.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
wwctl node set wwnode1 --tagadd IPXEMenuEntry=dracut
|
||||
|
||||
.. note::
|
||||
|
||||
The IPXEMenuEntry variable may be set at the node or profile level.
|
||||
|
||||
Alternatively, to direct GRUB to fetch the node's initramfs image and boot with
|
||||
dracut semantics, set a ``GrubMenuEntry`` tag for the node.
|
||||
|
||||
.. note::
|
||||
|
||||
Warewulf configures GRUB with a template located at
|
||||
``/etc/warewulf/grub/grub.cfg.ww``. Inspect the template to learn more about
|
||||
the dracut booting process.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
wwctl node set wwnode1 --tagadd GrubMenuEntry=dracut
|
||||
|
||||
.. note::
|
||||
|
||||
The ``GrubMenuEntry`` variable may be set at the node or profile level.
|
||||
|
||||
During boot, ``warewulfd`` will detect and dynamically serve an initramfs from a
|
||||
node's image in much the same way that it can serve a kernel from an image. This
|
||||
image is loaded by iPXE (or GRUB) which directs dracut to fetch the node's image
|
||||
during boot.
|
||||
|
||||
The wwinit module provisions to tmpfs. By default, tmpfs is permitted to use up
|
||||
to 50% of physical memory. This size limit may be adjusted using the kernel
|
||||
argument `wwinit.tmpfs.size`. (This parameter is passed to the `size` option
|
||||
during tmpfs mount. See ``tmpfs(5)`` for more details.)
|
||||
Reference in New Issue
Block a user