kelvin/warewulf
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

feat(docs): New documentation

Browse Source 
Convert docs to mkdocs, move back into repo, publish to repo
This commit is contained in:
Brian Clemens
2022-09-15 21:00:17 +09:00
committed by GitHub
parent 8c0710e412
commit 04ee8cc967
34 changed files with 2482 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
userdocs/contents/background.rst Normal file
View File

@@ -0,0 +1,18 @@
==========
Background
==========
Warewulf is based on the design of the original Beowulf Cluster design (and thus the name, soft\ **WARE** implementation of the beo\ **WULF**)
The `Beowulf Cluster <https://en.wikipedia.org/wiki/Beowulf_cluster>`_ design was developed in 1996 by Dr. Thomas Sterling and Dr. Donald Becker at NASA. The architecture is defined as a group of similar compute worker nodes all connected together using standard commodity equipment on a private network segment. The control node (historically referred to as the "master" or "head" node) is dual homed (has two network interface cards) with one of these network interface cards attached to the upstream public network and the other connected to a private network which connects to all of the compute worker nodes (as seen in the figure below).
.. image:: beowulf_architecture.png
:alt: Beowulf architecture
This simple topology is the foundation for creating every scalable HPC cluster resource. Even today, almost 30 years after the inception of this architecture, this is the baseline architecture that traditional HPC systems are built to.
Other considerations for a working HPC-type cluster are storage, scheduling and resource management, monitoring, interactive systems, etc. For smaller systems, much of these requirements can be managed all from a single control node, but as the system scales, it may need to have groups of nodes dedicated to these different services.
Warewulf is easily capable of building simple and turnkey HPC clusters, to giant massive complex multi-purpose computing clusters, through next generation computing platforms.
Anytime a cluster of systems is needed, Warewulf is your tool!

BIN
userdocs/contents/beowulf_architecture.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 91 KiB

136
userdocs/contents/configuration.rst Normal file
View File

@@ -0,0 +1,136 @@
======================
Warewulf Configuration
======================
The default installation of Warewulf will put all of the configuration
files into ``/etc/warewulf/``. In that directory, you will find the
primary configuration files needed by Warewulf.
warewulf.conf
=============
The Warewulf configuration exists as follows in the current version of
Warewulf (4.3.0):
.. code-block:: yaml
WW_INTERNAL: 43
ipaddr: 192.168.200.1
netmask: 255.255.255.0
network: 192.168.200.0
warewulf:
port: 9873
secure: false
update interval: 60
autobuild overlays: true
host overlay: true
syslog: false
dhcp:
enabled: true
range start: 192.168.200.50
range end: 192.168.200.99
systemd name: dhcpd
tftp:
enabled: true
systemd name: tftp
nfs:
enabled: true
export paths:
- path: /home
export options: rw,sync
mount options: defaults
mount: true
- path: /opt
export options: ro,sync,no_root_squash
mount options: defaults
mount: false
systemd name: nfs-server
Generally you can leave this file as is, as long as you set the
appropriate networking information. Specifically the following
configurations:
* ``ipaddr``: This is the control node's networking interface connecting
to the cluster's **PRIVATE** network. This configuration must match
the host's network IP address for the cluster's private interface.
* ``netmask``: Similar to the ``ipaddr``, this is the subnet mask for the
cluster's **PRIVATE** network and it must also match the host's
subnet mask for the cluster's private interface.
* ``dhcp:range start`` and ``dhcp:range end``: This address range must
exist in the network defined above. If it is outside of this
network, failures will occur. This specifies the range of addresses
you want DHCP to use.
.. note::
The network configuration listed above assumes the network
layout in the [Background](background.md) portion of the
documentation.
The other configuration options are usually not touched, but they are
explained as follows:
* ``*:enabled``: This disables Warewulf's control of an external
service. This is useful if you want to manage that service directly.
* ``*:systemd name``: This is so Warewulf can control some of the host's
services. For the distributions that we've built and tested this on,
these will require no changes.
* ``warewulf:port``: This is the port that the Warewulf web server will
be listening on. It is recommended not to change this so there is no
misalignment with node's expectations of how to contact the Warewulf
service.
* ``warewulf:secure``: When ``true``, this limits the Warewulf server to
only respond to runtime overlay requests originating from a
privileged report port. This makes it so that only the ``root`` user
on a compute node can request the runtime overlay. While generally
there is nothing super "secure" in these overlays, this adds the
necessary protection that the user's can not obtain this
information.
* ``warewulf:update interval``: This defines the frequency (in seconds)
with which the Warewulf client on the compute node fetches overlay
updates.
* ``warewulf:autobuild overlays``: This determines whether per-node
overlays will automatically be rebuilt, e.g., when an underlying
overlay is changed.
* ``warewulf:host overlay``: This determines whether the special ``host``
overlay is applied to the Warewulf server during configuration. (The
host overlay is used to configure the dependent services.)
* ``warewulf:syslog``: This determines whether Warewulf server logs go
to syslog or are written directly to a log file. (e.g.,
``/var/log/warewulfd.log``)
* ``nfs:export paths``: Warewulf will automatically set up the NFS
exports if you wish for it to do this.
nodes.conf
==========
The ``nodes.conf`` is the primary database file for all compute
nodes. It is a flat text YAML configuration file that is managed by
the ``wwctl`` command, but some sites manage the compute nodes and
infrastructure via configuration management. This file being flat text
and very light weight makes management of the node configurations very
easy no matter what your configuration paradigm is.
For the purpose of this document, we will not go into the detailed
format of this file as it is recommended to edit with the ``wwctl``
command.
.. note::
This configuration is not written at install time, but the
first time you attempt to run ``wwctl``, this file will be generated
if it does not exist already.
Directories
===========
The ``/etc/warewulf/ipxe/`` contains *text/templates* that are used by
the Warewulf configuration process to configure the ``ipxe`` service.

155
userdocs/contents/containers.rst Normal file
View File

@@ -0,0 +1,155 @@
====================
Container Management
====================
Since the inception of Warewulf over 20 years ago, Warewulf has used the model of the "Virtual Node File System" (VNFS) as a template image for the compute nodes. This is similar to a golden master image, except that the node file system exists within a directory on the Warewulf control node (e.g. a `chroot()`).
In hindsight, we've been using containers all along, but the buzzword just didn't exist. Over the last 5-6 years, the enterprise has created a lot of tooling and standards around defining, building, distributing, securing, and managing containers, so Warewulf (as of v4.0) now integrates directly within the container ecosystem to facilitate the process of VNFS image management.
If you are not currently leveraging the container ecosystem in any other way, you can still build your own chroot directories and use Warewulf as you always have.
It is important to understand that Warewulf is not running a container runtime on the nodes. While that is absolutely possible to run containers from the booted hosts, Warewulf is provisioning the container image to the bare metal and booting it. This container will be used as the base operating system and by default it will run stateless in memory. This means when you reboot the node, the node persists no information about Warewulf or how it booted.
Container Tools
===============
There are different container managment tools available. Docker is probably the most recognizable one in the enterprise. Podman is another one that is gaining traction on the RHEL platforms. In HPC, Singularity is the most utilized container management tool. You can use any of these to create and manage the containers to be later imported into Warewulf.
Importing From A Registry
=========================
Warewulf supports importing an image from any OCI compliant registry. This means you can import from a public registry or from a private registry.
Here is an example of importing from Docker Hub.
.. code-block:: bash
$ sudo wwctl container import docker://warewulf/rocky rocky-8
Getting image source signatures
Copying blob d7f16ed6f451 done
Copying config da2ca70704 done
Writing manifest to image destination
Storing signatures
[LOG] info unpack layer: sha256:d7f16ed6f45129c7f4adb3773412def4ba2bf9902de42e86e77379a65d90a984
Updating the container's /etc/resolv.conf
Building container: rocky-8
.. note::
Most containers in Docker Hub are not "bootable", in that, they have a limited version of Systemd to make them lighter weight for container purposes. For this reason, don't expect any base Docker container (e.g. ``docker://rockylinux`` or ``docker://debian``) to boot properly. They will not, as they will get stuck into a single user mode. The containers in `https://hub.docker.com/u/warewulf <https://hub.docker.com/u/warewulf>`_ are not limited and thus they boot as you would expect.
Private Registry
----------------
It is possible to use a private registry that is password protected or does not have the requirement for TLS. In order to do so, you have two choices for handling the credentials.
* Set environmental variables
* Use ``docker login`` or ``podman login`` which will store the credentials locally
Please note, there is no requirement to install and use docker or podman on your control node just for importing images into Warewulf.
Here are the environmental variables that can be used.
.. code-block:: bash
WAREWULF_OCI_USERNAME
WAREWULF_OCI_PASSWORD
WAREWULF_OCI_NOHTTPS
Here is an example:
.. code-block:: bash
export WAREWULF_OCI_USERNAME=privateuser
export WAREWULF_OCI_PASSWORD=super-secret-password-or-token
sudo -E wwctl import docker://ghcr.io/privatereg/rocky:8
The above is just an example. Consideration should be done before doing it this way if you are in a security sensitive environment or shared environments. You would not want these showing up in bash history or logs.
Listing All Imported Containers
===============================
Once the container has been imported, you can list them all with the following command:
.. code-block:: bash
$ sudo wwctl container list
CONTAINER NAME BUILT NODES
rocky-8 true 0
Once a container has been imported and showing up in this list you can configure it to boot compute nodes.
Making Changes To Containers
============================
Warewulf has a minimal container runtime built into it. This means you can run commands inside of any of the containers and make changes to them as follows:
.. code-block:: bash
$ sudo wwctl container exec rocky-8 /bin/sh
[rocky-8] Warewulf> cat /etc/rocky-release
Rocky Linux release 8.4 (Green Obsidian)
[rocky-8] Warewulf> exit
Rebuilding container...
[INFO] Skipping (VNFS is current)
You can also ``--bind`` directories from your host into the container when using the exec command. This works as follows:
.. code-block:: bash
$ sudo wwctl container exec --bind /tmp:/mnt rocky-8 /bin/sh
[rocky-8] Warewulf>
.. note::
As with any mount command, both the source and the target must exist. This is why the example uses the ``/mnt/`` directory location, as it is almost always present and empty in every Linux distribution (as prescribed by the LSB file hierarchy standard).
When the command completes, if anything within the container changed, the container will be rebuilt into a bootable static object automatically.
Creating Containers From Scratch
================================
You can also create containers from scratch and import those containers into Warewulf as previous versions of Warewulf did.
Building A Container From Your Host
-----------------------------------
RPM based distributions, as well as Debian variants can all bootstrap mini ``chroot()`` directories which can then be used to bootstrap your node's container.
For example, on an RPM based Linux distribution with YUM or DNF, you can do something like the following:
.. code-block:: bash
$ sudo yum install --installroot /tmp/newroot basesystem bash \
chkconfig coreutils e2fsprogs ethtool filesystem findutils \
gawk grep initscripts iproute iputils net-tools nfs-utils pam \
psmisc rsync sed setup shadow-utils rsyslog tzdata util-linux \
words zlib tar less gzip which util-linux openssh-clients \
openssh-server dhclient pciutils vim-minimal shadow-utils \
strace cronie crontabs cpio wget rocky-release ipmitool yum \
NetworkManager
You can do something similar with Debian-based distributions:
.. code-block:: bash
sudo apt-get install debootstrap
sudo debootstrap stable /tmp/newroot http://ftp.us.debian.org/debian
Once you have created and modified your new ``chroot()``, you can import it into Warewulf with the following command:
.. code-block:: bash
sudo wwctl container import /tmp/newroot containername
Building A Container Using Singularity
--------------------------------------
Singularity, a container platform for HPC and performance intensive applications, can also be used to create node containers for Warewulf. There are several Singularity container recipes in the ``containers/Singularity/`` directory and can be found on GitHub at `https://github.com/hpcng/warewulf/tree/main/containers/Singularity <https://github.com/hpcng/warewulf/tree/main/containers/Singularity>`_.
You can use these as starting points and adding any additional steps you want in the ``%post`` section of the recipe file. Once you've done that, installing Singularity, building a container sandbox and importing into Warewulf can be done with the following steps:
.. code-block:: bash
sudo yum install epel-release
sudo yum install singularity
sudo singularity build --sandbox /tmp/newroot /path/to/Singularity/recipe.def
sudo wwctl container import /tmp/newroot containername

45
userdocs/contents/initialization.rst Normal file
View File

@@ -0,0 +1,45 @@
=======================
Warewulf Initialization
=======================
System Services
===============
Once Warewulf has been installed and configured, it is ready to be initialized and have the associated system services started. To do this, start by configuring the system services that Warewulf depends on for operation. To do that, run the following command:
.. code-block:: bash
sudo wwctl configure --all
This command will configure the system for Warewulf to run properly. Here are the things it will do:
* **dhcp**: (re)Write the DHCP configuration and restart the service from the configured template in ``/etc/warewulf/dhcp/`` and enable the system service.
* **hostfile**: Update the system's /etc/hosts file based on the template ``/etc/warewulf/hosts.tmpl``.
* **nfs**: Configure the NFS server on the control node as well as the ``/etc/fstab`` in the system overlay based on the configuration in ``/etc/warewulf/warewulf.conf`` and enable the system service.
* **ssh**: Create the appropriate host keys (stored in ``/etc/warewulf/keys/``) and user keys for passwordless ``ssh`` into the nodes.
* **tftp**: Write the appropriate binary PXE/iPXE blobs to the TFTP root directory and enable the system service.
This command will quickly setup the system services per the Warewulf configuration. Watch this output carefully for errors and resolve them in the configuration portion of this manual.
Warewulf Service
================
The Warewulf installation attempts to register the Warewulf service with Systemd, so it should be as easy to start/stop/check as any other Systemd service:
.. code-block:: bash
sudo systemctl enable --now warewulfd
You can also check and control the Warewulf service using the `wwctl` command line program as follows:
.. code-block:: bash
sudo wwctl server status
.. note::
If the Warewulf service is running via Systemd, restarting stopping, and starting it from the ``wwctl`` command may result in unexpected results.
Logs
----
The Warewulf server logs are by default written to ``/var/log/warewulfd.log``.

79
userdocs/contents/installation.rst Normal file
View File

@@ -0,0 +1,79 @@
#####################
Warewulf Installation
#####################
There are multiple methods to install Warewulf, this page describes the installation process of multiple methods:
Binary RPMs
===========
While the Warewulf project does not build binary RPMs, you can obtain them from `CIQ <https://www.ctrliq.com/>`_ and use them for non-production use from their public YUM and DNF repositories at: `https://repo.ctrliq.com <https://repo.ctrliq.com/>`_
This is the easiest method to install Warewulf and can be done as follows:
.. code-block:: bash
sudo yum install -y https://repo.ctrliq.com/rhel/8/ciq-release.rpm
sudo yum install -y warewulf
> note: as mentioned, these binaries are part of CIQ's commercial support offering but they can be used for non-production and testing uses. If you are interested in using these binaries for production, please contact CIQ at: `info@ctrliq.com <mailto:info@ctrliq.com>`_.
Compiled Source code
====================
Before you build the Warewulf source code you will first need to install the build dependencies:
* ``make``: This should be available via your Linux distribution's package manager (e.g. ``dnf install make``)
* ``go``: Golang is also available on most current Linux distributions, but if you wish to install the most recent version, you can find that here: `https://golang.org/dl/ <https://golang.org/dl/>`_
* Depending on your Linux Distribution, you may need to install other development packages. Typically it is recommended to install the entire development group like ``dnf groupinstall "Development Tools"``
Once these dependencies are installed, you can obtain and build the source code as follows:
Release Tarball
---------------
When the Warewulf project releases stable versions, they are available via source form here:
`https://github.com/hpcng/warewulf/tags <https://github.com/hpcng/warewulf/tags/>`_
Select the version you wish to install and download the tarball to any location on the server, then follow these directions making the appropriate substitutions:
.. code-block:: bash
# EDIT HERE
VERSION=4.2.0
DOWNLOAD=/tmp/warewulf-${4.2.0}.tar.gz
# COPY/PASTE THIS
mkdir ~/src
cd ~/src
tar xvf ${DOWNLOAD}
cd warewulf-${VERSION}
make all && sudo make install
Git
---
Warewulf is developed in "Git", a source code management platform that allows collaborative development and revision control. From the Git repository, you can download different versions of the project either from tags or branches. By default, when you go to the GitHub page, you will find the default branch entitled ``main``. The ``main`` branch is where most of the active development occurs, so if you want to obtain the latest and greatest version of Warewulf, this is where to go. But be forewarned, using a snapshot from ``main`` is not guaranteed to be stable or generally supported for production. If you are building for production, it is best to download a release tarball from the main site, the GitHub releases page, or from a Git tag.
.. code-block:: bash
mkdir ~/git
cd ~/git
git clone https://github.com/hpcng/warewulf.git
cd warewulf
git checkout main # or switch to a tag like '4.2.0'
make all && sudo make install
Runtime Dependencies
--------------------
In Warewulf's default configuration, it will require some operating system provided services. Generally these are provided by your installation vendor and can be installed over the network.
These are the services you will need to install:
* ``dhcp-server``
* ``tftp-server``
* ``nfs-utils``
If you are using an Enterprise Linux compatible distribution you can install them with: ``yum install dhcp-server tftp-server nfs-utils``

36
userdocs/contents/introduction.rst Normal file
View File

@@ -0,0 +1,36 @@
============
Introduction
============
The Warewulf Vision
===================
Warewulf has had a number of iterations over the last 20 years, but its design tenets have always remained the same: a simple, scalable, stateless (however some versions were able to provision stateful), and very flexible operating system provisioning system for all types of clusters. This is an overview of how Warewulf works.
About Warewulf
==============
Warewulf is an operating system provisioning platform for Linux that is designed to produce secure, scalable, turnkey cluster deployments that maintain flexibility and simplicity.
Since its initial release in 2001, Warewulf has become the most popular open source and vendor-agnostic provisioning system within the global HPC community. Warewulf is known for its massive scalability and simple management of stateless (disk optional) provisioning.
Warewulf leverages a simple administrative model centralizing administration around virtual node images which are used to provision out to the cluster nodes. This means you can have hundreds or thousands of cluster nodes all booting and running on the same, identical virtual node file system image. As of Warewulf v4, the virtual node image is a standard container image which means all compute resources within a cluster can be managed using any existing container tooling and/or CI pipelines that are being used. This can be as simple as DockerHub or your own private GitLab CI infrastructure.
With this architecture, Warewulf combines the best of High Performance Computing (HPC), Cloud, Hyperscale, and Enterprise deployment principals to create and maintain large scalable stateless clusters.
While Warewulf's roots are in HPC, it has been used for many different types of tasks and use cases. Everything from clustered web servers, to rendering farms, and even Kubernetes and cloud deployments, Warewulf brings benefit in experience of general operating system management at scale.
Features
========
* **Lightweight**: Warewulf provisions stateless operating system images and then gets out of the way. There should be no underlying system dependencies or changes to the provisioned cluster node operating systems.
* **Simple**: Warewulf is used by hobbyists, researchers, scientists, engineers and systems administrators because it is easy, lightweight, and simple.
* **Flexible**: Warewulf is highly flexible and can address the needs of any environment-- from a computer lab with graphical workstations, to under-the-desk clusters, to massive supercomputing centers providing traditional HPC capabilities to thousands of users.
* **Agnostic**: From the Linux distribution of choice to the underlying hardware, Warewulf is agnostic and standards compliant. From ARM to x86, Atos to Dell, Debian, SuSE, Rocky, CentOS, and RHEL, Warewulf can do it all.
* **Secure**: Warewulf is the only stateless provisioning system that will support SELinux out of the box. Just enable your node operating system container to support SELinux, and Warewulf do the rest!
* **Open Source**: For the last 20 years, Warewulf has remained open source and continues to be the golden standard for cluster provisioning.

145
userdocs/contents/ipmi.rst Normal file
View File

@@ -0,0 +1,145 @@
====
IPMI
====
It is possible to control the power or connect a console to your nodes being managed by Warewulf by connecting to the BMC through the use of IPMI. We will discuss how to set this up below.
IPMI Settings
=============
The common settings for the IPMI interfaces on all nodes can be set on a Profile level. The only field that would need to be assigned to each individual node would be the IP address.
If an individual node has different settings, you can set the IPMI settings for that specific node, overriding the default settings.
Here is a table outlining the fields on a Profile and Node level, along with the parameters that can be used when running ``wwctl profile set`` or ``wwctl node set``.
============= =============== ======== ======= ================== =============
Field Parameter Profile Node Valid Values Default Value
============= =============== ======== ======= ================== =============
IpmiIpaddr --ipmi X
IpmiNetmask --ipminetmask X X
IpmiPort --ipmiport X X 623
IpmiGateway --ipmigateway X X
IpmiUserName --ipmiuser X X
IpmiPassword --ipmipass X X
IpmiInterface --ipmiinterface X X 'lan' or 'lanplus' lan
============= =============== ======== ======= ================== =============
Reviewing Settings
==================
There are multiple ways to review the IPMI settings. They can be reviewed from a Profile level, all the way down to a specific Node.
Profile View
------------
.. code-block:: bash
$ sudo wwctl profile list -a
################################################################################
PROFILE NAME FIELD VALUE
default Id default
default Comment This profile is automatically included for each node
default Cluster --
default Container rocky
default Kernel 4.18.0-348.2.1.el8_5.x86_64
default KernelArgs --
default Init --
default Root --
default RuntimeOverlay --
default SystemOverlay --
default Ipxe --
default IpmiNetmask 255.255.255.0
default IpmiPort --
default IpmiGateway 192.168.99.1
default IpmiUserName admin
default IpmiInterface lanplus
default eth0:IPADDR --
default eth0:NETMASK 255.255.240.0
default eth0:GATEWAY 10.1.96.6
default eth0:HWADDR --
default eth0:TYPE --
default eth0:DEFAULT false
Node View
---------
.. code-block:: bash
$ sudo wwctl node list node0001 -a
################################################################################
NODE FIELD PROFILE VALUE
node0001 Id -- node0001
node0001 Comment default This profile is automatically included for each node
node0001 Cluster -- --
node0001 Profiles -- default
node0001 Discoverable -- false
node0001 Container default rocky
node0001 Kernel default 4.18.0-348.2.1.el8_5.x86_64
node0001 KernelArgs -- (quiet crashkernel=no vga=791 rootfstype=rootfs)
node0001 RuntimeOverlay -- (default)
node0001 SystemOverlay -- (default)
node0001 Ipxe -- (default)
node0001 Init -- (/sbin/init)
node0001 Root -- (initramfs)
node0001 IpmiIpaddr -- 192.168.99.10
node0001 IpmiNetmask -- 255.255.255.0
node0001 IpmiPort -- --
node0001 IpmiGateway -- 192.168.99.1
node0001 IpmiUserName default admin
node0001 IpmiInterface default lanplus
node0001 eth0:HWADDR -- 52:54:00:1a:08:60
node0001 eth0:IPADDR -- 192.168.100.152
node0001 eth0:NETMASK default 255.255.255.0
node0001 eth0:GATEWAY default 192.168.100.1
node0001 eth0:TYPE -- --
node0001 eth0:DEFAULT -- false
Review Only IPMI Settings
-------------------------
The above views show you everything that is set on a Profile or Node level. That is a lot of detail. If you want to view key IPMI connecton details for a node or all nodes, you can do the following.
.. code-block:: bash
$ sudo wwctl node list -i
NODE NAME IPMI IPADDR IPMI PORT IPMI USERNAME IPMI PASSWORD IPMI INTERFACE
============================================================================================================
node0001 192.168.99.10 -- admin supersecret lanplus
node0002 192.168.99.11 -- admin supersecret lanplus
node0003 192.168.99.12 -- admin supersecret lanplus
Power Commands
==============
The ``power`` command can be used to manage the current power state of your nodes through IPMI.
``wwctl power [command]`` where ``[command]`` is one of:
cycle
Turns the power off and then on
off
Turns the power off
on
Turns the power on
reset
Issues a reset
soft
Shutdown gracefully
status
Shows current power status
Console
=======
If your node is setup to use serial over lan (SOL), Warewulf can connect a console to the node.
``sudo wwctl node console node0001``

53
userdocs/contents/kernel.rst Normal file
View File

@@ -0,0 +1,53 @@
=================
Kernel Management
=================
Node Kernels
============
Warewulf nodes require a Linux kernel to boot. There are multiple ways to do this, but the default, and easiest way is to install the kernel you wish to use for a particular container, into the container.
Warewulf will locate the kernel automatically within the container and by default use that kernel for any node configured to use that container image.
You can see what kernel is included in a container by using the ``wwctl container list`` command:
.. code-block:: bash
$ sudo wwctl container list
CONTAINER NAME NODES KERNEL VERSION
alpine 0
rocky 0 4.18.0-348.12.2.el8_5.x86_64
rocky_updated 1 4.18.0-348.23.1.el8_5.x86_64
Here you will notice the alpine contianer that was imported has no kernel within it, and each of the rocky containers include a kernel.
This model was introduced in Warewulf 4.3.0. Previously, Warewulf managed the kernel and the container separately, which made it hard to build and distribute containers that have custom drivers and/or configurations included (e.g. OFED, GPUs, etc.).
Kernel Overrides
================
It is still possible to specify a kernel to a container if it doesn't include it's own kernel, or if you wish to override the default kernel by using the ``kernel override`` capability.
You can specify this option either within the ``nodes.conf`` directly or via the command line with the ``--kerneloverride`` option to ``wwctl node set`` or ``wwctl profile set`` commands.
In this case you will also need to import a kernel specifically into Warewulf for this purpose using the ``wwctl kernel import`` command as follows:
.. code-block:: bash
$ sudo wwctl kernel import $(uname -r)
4.18.0-305.3.1.el8_4.x86_64: Done
This process will import not only the kernel image itself, but also all of the kernel modules and firmware associated to this kernel.
Listing All Imported Kernels
----------------------------
Once the kernel has been imported, you can list them all with the following command:
.. code-block:: bash
$ sudo wwctl kernel list
VNFS NAME NODES
4.18.0-305.3.1.el8_4.x86_64 0
Once a kernel has been imported and showing up in this list you can configure it to boot compute nodes.

202
userdocs/contents/nodeconfig.rst Normal file
View File

@@ -0,0 +1,202 @@
==================
Node Configuration
==================
The Node Configuration DB
=========================
As mentioned in the [Configuration](configuration) section, node
configs are persisted to the ``nodes.conf`` YAML file, but generally it
is best not to edit this file directly (however that is supported, it
is just prone to errors).
This method of using a YAML configuration file as a backend datastore
is both scalable and very lightweight. We've tested this out to over
10,000 node entries which yielded update latencies under 1 second,
which we felt was both tolerable and advantageous.
Adding a New Node
=================
Creating a new node is as simple as running the following command:
.. code-block:: bash
$ sudo wwctl node add n0000
Added node: n0000
Node Names
----------
For small clusters, you can use simple names (e.g. ``n0000``); but for
larger, more complicated clusters that are comprised of multiple
clusters and roles it is highly recommended to use node names that
include a cluster descriptor. In Warewulf, this is generally done by
using a domain name (e.g. ``n0000.cluster01``). Warewulf will
automatically assume that the domain is the equivalent of the cluster
name.
This also means that you can address groups of nodes by the cluster
descriptor with globs. For example, you are able to refer to all nodes
in "cluster01" with the following string: ``*.cluster01`` which is
valuable for other ``wwctl`` commands.
Listing Nodes
=============
Once you have configured one or more nodes, you can list them and
their attributes as follows:
.. code-block:: bash
$ sudo wwctl node list
NODE NAME PROFILES NETWORK
================================================================================
n0000 default
You can also see the node's full attribute list by specifying the ``-a``
option (all):
.. code-block:: bash
$ sudo wwctl node list -a
################################################################################
NODE FIELD PROFILE VALUE
n0000 Id -- n0000
n0000 Comment default This profile is automatically included for each node
n0000 Cluster -- --
n0000 Profiles -- default
n0000 Discoverable -- false
n0000 Container -- --
n0000 KernelOverride -- --
n0000 KernelArgs -- (quiet crashkernel=no vga=791 rootfstype=rootfs)
n0000 RuntimeOverlay -- (default)
n0000 SystemOverlay -- (default)
n0000 Ipxe -- (default)
n0000 Init -- (/sbin/init)
n0000 Root -- (initramfs)
n0000 IpmiIpaddr -- --
n0000 IpmiNetmask -- --
n0000 IpmiPort -- --
n0000 IpmiGateway -- --
n0000 IpmiUserName -- --
n0000 IpmiInterface -- --
n0000 IpmiWrite -- --
.. note::
The attribute values in parenthesis are default values and can
be overridden in the next section, granted, the default values are
generally usable.
Setting Node Attributes
=======================
In the above output we can see that there is no kernel or container
defined for this node. To provision a node, the minimum requirements
are a kernel and container, and for that node to be useful, we will
also need to configure the network so the nodes are reachable after
they boot.
Node configurations are set using the ``wwctl node set`` command. To see
a list of all configuration attributes, use the command ``wwctl node
set --help``.
Configuring the Node's Container Image
======================================
.. code-block:: bash
$ sudo wwctl node set --container rocky-8 n0000
Are you sure you want to modify 1 nodes(s): y
And you can check that the container name is set for ``n0000``:
.. code-block:: bash
$ sudo wwctl node list -a n0000 | grep Container
n0000 Container -- rocky-8
Configuring the Node's Kernel
-----------------------------
While the recommended method for assigning a kernel in 4.3 and beyond
is to include it in the container / node image, a kernel can still be
specified as an override at the node or profile.
.. code-block:: bash
$ sudo wwctl node set --kerneloverride $(uname -r) n0000
Are you sure you want to modify 1 nodes(s): y
$ sudo wwctl node list -a n0000 | grep KernelOverride
n0000 KernelOverride -- 4.18.0-305.3.1.el8_4.x86_64
Configuring the Node's Network
------------------------------
To configure the network, we have to pick a network device name and
provide the network information as follows:
.. code-block:: bash
$ sudo wwctl node set --netdev eth0 --hwaddr 11:22:33:44:55:66 --ipaddr 10.0.2.1 --netmask 255.255.252.0 n0000
Are you sure you want to modify 1 nodes(s): y
You can now see that the node contains configuration attributes for
container, kernel, and network:
.. code-block:: bash
$ sudo wwctl node list -a n0000
################################################################################
NODE FIELD PROFILE VALUE
n0000 Id -- n0000
n0000 Comment default This profile is automatically included for each node
n0000 Cluster -- --
n0000 Profiles -- default
n0000 Discoverable -- false
n0000 Container -- rocky-8
n0000 Kernel -- 4.18.0-305.3.1.el8_4.x86_64
n0000 KernelArgs -- (quiet crashkernel=no vga=791 rootfstype=rootfs)
n0000 RuntimeOverlay -- (default)
n0000 SystemOverlay -- (default)
n0000 Ipxe -- (default)
n0000 Init -- (/sbin/init)
n0000 Root -- (initramfs)
n0000 IpmiIpaddr -- --
n0000 IpmiNetmask -- --
n0000 IpmiPort -- --
n0000 IpmiGateway -- --
n0000 IpmiUserName -- --
n0000 IpmiInterface -- --
n0000 default:DEVICE -- eth0
n0000 default:HWADDR -- 11:22:33:44:55:66
n0000 default:IPADDR -- 10.0.2.1
n0000 default:NETMASK -- 255.255.252.0
n0000 default:GATEWAY -- --
n0000 default:TYPE -- --
n0000 default:DEFAULT -- false
Un-setting Node Attributes
==========================
If you wish to ``unset`` a particular value, set the value to
``UNDEF``. For example:
.. code-block:: bash
$ sudo wwctl node set --cluster cluster01 n0000
Are you sure you want to modify 1 nodes(s): y
$ sudo wwctl node list -a n0000 | grep Cluster
n0000 Cluster -- cluster01
And to unset this configuration attribute:
.. code-block:: bash
$ sudo wwctl node set --cluster UNDEF n0000
Are you sure you want to modify 1 nodes(s): y
$ sudo wwctl node list -a n0000 | grep Cluster
n0000 Cluster -- --

141
userdocs/contents/overlays.rst Normal file
View File

@@ -0,0 +1,141 @@
=================
Warewulf Overlays
=================
So at this point, we have discussed how Warewulf is designed to scalably provision and manage thousands of cluster nodes by utilizing identical stateless boot images. And there-in lies a problem to solve. If these boot images are completely identical, then how do we configure things like hostnames? IP addresses? Or any other node specific custom configurations?
While some of this can be managed by services like DHCP, and other bits by configuration management, which can absolutely be done with Warewulf and many people choose to do, these are heavy-weight solutions to a simple problem to solve.
Warewulf solves this with overlays and uses overlays in different ways through the provisioning process. Two of these overlays are exposed to the users, the **system overlay** and the **runtime overlay**.
.. note::
Another overlay that isn't directly exposed is the **kmods overlay** which contains all of the kernel modules to match the configured kernel. Because this overlay is used "behind the scenes" it is outside the scope of this document.
System Overlay
==============
The System Overlay is used by the core of Warewulf to setup the environment on the node necessary for provisioning. The default system overlay is called ``wwinit``. Generally speaking, it will not be necessary to make changes to this overlay, but it is possible to change or configure this overlay to meet site specific needs if necessary.
Runtime Overlay
===============
The Runtime Overlay is the overlay that is responsible for most of the typical system administration configurations. Here you will make changes necessary to support your operating system as well as application configurations.
Once the system is provisioned and booted, the ``wwclient`` program (which is provisioned as part of the ``wwinit`` system overlay) will continuously update the node with updates in the runtime overlay.
Templates
=========
Templates allow you to create dynamic content such that the files downloaded for each node will be customized for that node. Templates allow you to insert everything from variables, to including files from the control node, as well as conditional content and loops.
Warewulf uses the ``text/template`` engine to facilitate implementing dynamic content in a simple and standardized manner.
All template files will end with the suffix of ``.ww``. That tells Warewulf that when building a file, that it should parse that file as a template. When it does that, the resulting file is static and can have node customizations that are obtained from the node configuration attributes.
.. note::
When the file is persisted within the built overlay, the ``.ww`` will be dropped, so ``/etc/hosts.ww`` will end up being ``/etc/hosts``.
Using Overlays
==============
Warewulf includes a command group for manipulating overlays (``wwctl overlay``). With this you can add, edit, remove, change ownership, permissions, etc.
The general syntax is as follows:
.. code-block:: bash
wwctl overlay [action] [overlay name] ...
* **action**: the overlay subcommand you are invoking
* **overlay name**: the name of the overlay in question within a given type
* **...**: additional arguments are action specific
By default there is one overlay in each of the system and runtime overlay types. Both overlays are called "default". To say it differently, there are two default overlays, one is a system overlay and one is a runtime overlay.
Viewing the Files Within an Overlay
===================================
Overlays can be viewed with the command ``wwctl overlay list``. You can see the files within an overlay by adding the ``-a`` or ``-l`` options as follows:
.. code-block:: bash
$ sudo wwctl overlay list -l generic
PERM MODE UID GID OVERLAY FILE PATH
-rwxr-xr-x 0 0 generic /etc/
-rw-r--r-- 0 0 generic /etc/group.ww
-rw-r--r-- 0 0 generic /etc/hosts.ww
-rw-r--r-- 0 0 generic /etc/passwd.ww
-rwxr-xr-x 0 0 generic /root/
-rwxr-xr-x 0 0 generic /root/.ssh/
-rw-r--r-- 0 0 generic /root/.ssh/authorized_keys.ww
Creating a New File Within an Overlay
=====================================
Just like any file on the system, you can create and edit a file at the same time. So to do that, you simple ``edit`` a new file as follows:
.. code-block:: bash
$ sudo wwctl overlay edit [overlay name] [file path]
For example:
.. code-block:: bash
$ sudo wwctl overlay edit generic /etc/testfile
and you can validate that the file is there with the ``list`` command:
.. code-block:: bash
$ sudo wwctl overlay list generic -l
PERM MODE UID GID RUNTIME-OVERLAY FILE PATH
-rwxr-xr-x 0 0 generic /etc/
-rw-r--r-- 0 0 generic /etc/group.ww
-rw-r--r-- 0 0 generic /etc/hosts.ww
-rw-r--r-- 0 0 generic /etc/passwd.ww
-rwxr-xr-x 0 0 generic /etc/testfile
-rwxr-xr-x 0 0 generic /root/
-rwxr-xr-x 0 0 generic /root/.ssh/
-rw-r--r-- 0 0 generic /root/.ssh/authorized_keys.ww
.. note::
To create a template file, simply name the file with the suffix ``.ww``. This suffix will tell Warewulf that the file should be parsed by the templating engine and written into the overlay with the suffix stripped off.
Building Overlays
=================
By default Warewulf will build/update and cache overlays as needed (configurable in the ``warewulf.conf``).
You can however build overlays by hand, and in some cases this will be advantageous (like if you are freshly booting thousands of compute nodes in parallel). The command to do that is:
.. code-block:: bash
# wwctl overlay build n00[00-10]
Building overlays for n0000: [wwinit, generic]
Building overlays for n0001: [wwinit, generic]
Building overlays for n0002: [wwinit, generic]
Building overlays for n0003: [wwinit, generic]
Building overlays for n0004: [wwinit, generic]
Building overlays for n0005: [wwinit, generic]
Building overlays for n0006: [wwinit, generic]
Building overlays for n0007: [wwinit, generic]
Building overlays for n0008: [wwinit, generic]
Building overlays for n0009: [wwinit, generic]
Building overlays for n0010: [wwinit, generic]
Other Overlay Actions
=====================
Warewulf includes a number of overlay action commands to interact with the overlays in a programmatic and controlled manner. All of the commands use very similar usage structure and work as the above examples do. A summary of all of the overlay actions are as follows:
* **build**: (Re)build an overlay
* **chmod**: Change file permissions within an overlay
* **chown**: Change file ownership within an overlay
* **create**: Initialize a new Overlay
* **delete**: Delete Warewulf Overlay or files
* **edit**: Edit/Create a file within a Warewulf Overlay
* **import**: Import a file into a Warewulf Overlay
* **list**: List Warewulf Overlays and files
* **mkdir**: Create a new directory within an Overlay
* **show**: Show (cat) a file within a Warewulf Overlay

162
userdocs/contents/profiles.rst Normal file
View File

@@ -0,0 +1,162 @@
#############
Node Profiles
#############
Profiles provide a way to scalably group node configurations together. Instead of redundant configurations for each node, you can put that into a profile and the nodes will inherit these configurations. This is very handy if you have groups of node specific customizations. For example, a few hundred nodes that are running a particular container or kernel, and another group of nodes that are running a different kernel or container.
Any node configuration attributes can be applied to a profile, but there are always going to be some node configurations which must be specific to a node, like a network HW/MAC address or an IP address.
An Introduction To Profiles
==============================
Every new node is automatically added to a profile called ``default``. You can view the configuration attributes of this profile by using the ``wwctl profile list`` command. Like the ``wwctl node list`` command, this will provide a summary, but you can see **all** configuration attributes by using the ``--all`` or ``-a`` flag as follows:
.. code-block:: bash
$ sudo wwctl profile list
PROFILE NAME COMMENT/DESCRIPTION
================================================================================
default This profile is automatically included for each node
And with the ``-a`` flag:
.. code-block:: bash
$ sudo wwctl profile list -a
################################################################################
PROFILE NAME FIELD VALUE
default Id default
default Comment This profile is automatically included for each node
default Cluster --
default Container --
default Kernel --
default KernelArgs --
default Init --
default Root --
default RuntimeOverlay --
default SystemOverlay --
default Ipxe --
default IpmiIpaddr --
default IpmiNetmask --
default IpmiPort --
default IpmiGateway --
default IpmiUserName --
default IpmiInterface --
As you can see here, there is only one attribute set by default in this profile, and that is the "Comment" field. That Comment is inherited by any nodes that are configured to use this profile. So if we look at the node we configured in the last section, we can see that configuration attribute there:
.. code-block:: bash
$ sudo wwctl node list -a | head -n 5
################################################################################
NODE FIELD PROFILE VALUE
n0000 Id -- n0000
n0000 Comment default This profile is automatically included for each node
n0000 Cluster -- --
Here you can see that the "Comment" attribute was inherited by this node, and it also provides you with the information of which profile this attribute was inherited from. This is very useful information as nodes can be part of multiple profiles with inheritance being cascading.
Multiple Profiles
=================
For demonstration purposes, let's create another profile and demonstrate how to use this second profile.
.. code-block:: bash
$ sudo wwctl profile add test_profile
$ sudo wwctl profile list
PROFILE NAME COMMENT/DESCRIPTION
================================================================================
default This profile is automatically included for each node
test_profile --
Now that we've created a new profile, let's create a configuration attribute in this profile:
.. code-block:: bash
$ sudo wwctl profile set --cluster cluster01 test_profile
? Are you sure you want to modify 1 profile(s)? [y/N] y█
$ sudo wwctl profile list -a test_profile | grep Cluster
test_profile Cluster cluster01
Lastly we just need to configure this profile to our node(s):
.. code-block:: bash
$ sudo wwctl node set --addprofile test_profile n0000
Are you sure you want to modify 1 nodes(s): y
And you can now verify that the node has both profile configurations:
.. code-block:: bash
$ sudo wwctl node list -a | head -n 6
################################################################################
NODE FIELD PROFILE VALUE
n0000 Id -- n0000
n0000 Comment default This profile is automatically included for each node
n0000 Cluster test_profile cluster01
n0000 Profiles -- default,test_profile
Cascading Profiles
==================
In the previous example, we set a single node to have two profile configurations. We can also overwrite configurations as follows:
.. code-block:: bash
$ sudo wwctl profile set --comment "test comment" test_profile
Are you sure you want to modify 1 profile(s): y
$ sudo wwctl node list -a | head -n 6
################################################################################
NODE FIELD PROFILE VALUE
n0000 Id -- n0000
n0000 Comment test_profile test comment
n0000 Cluster test_profile cluster01
n0000 Profiles -- default,test_profile
And if we delete the superseded profile attribute from ``test_profile`` we can now see the previous configuration:
.. code-block:: bash
$ sudo wwctl profile set --comment UNDEF test_profile
Are you sure you want to modify 1 profile(s): y
$ sudo wwctl node list -a | head -n 6
################################################################################
NODE FIELD PROFILE VALUE
n0000 Id -- n0000
n0000 Comment default This profile is automatically included for each node
n0000 Cluster test_profile cluster01
n0000 Profiles -- default,test_profile
This is a very useful feature for dealing with many groups of cluster nodes and/or testing new configurations on smaller subsets of cluster nodes. For example, you can use this method to run a different kernel on only a subset or group of cluster nodes without changing any other node attributes.
Overriding Profiles
===================
All profile configurations can be overwritten by a node configuration as can be seen here:
.. code-block:: bash
$ sudo wwctl node set --comment "This value takes precedent" n0000
Are you sure you want to modify 1 nodes(s): y
$ sudo wwctl node list -a | head -n 6
################################################################################
NODE FIELD PROFILE VALUE
n0000 Id -- n0000
n0000 Comment SUPERSEDED This value takes precedent
n0000 Cluster test_profile cluster01
n0000 Profiles -- default,test_profile
How To Use Profiles Effectively
===============================
There are a lot of ways to use profiles to facilitate the management of large cluster node attributes, but there is nothing inherent in the design of Warewulf that requires use of them for anything. It is completely reasonable to not use profiles at all to help with node configuration attributes.
But if you do wish to use profiles, the best way to use them is to manage "fixed" configurations of groups of cluster nodes. For example, if you have multiple sub-clusters in your cluster, it might be advantageous to have a ``cluster_name`` profile which includes things like network configurations, and/or a specific kernel, container, boot arguments, etc.
Node specific information, like HW/MAC addresses and IP addresses should always be put in a node configuration rather than a profile configuration.

43
userdocs/contents/provisioning.rst Normal file
View File

@@ -0,0 +1,43 @@
=================
Node Provisioning
=================
Once the nodes are configured in Warewulf, they are ready to boot.
Node Hardware Setup
===================
The only thing that Warewulf requires to provision is that the node is set to PXE boot. You may need to change the boot order if there is a local disk present and bootable. This is a configuration change you will have to make in the BIOS of the cluster node.
Each vendor does this differently and as a result we won't go into the setup specifics here and if you can not find information on how to PXE boot your nodes, please contact your hardware vendor support.
.. note::
If you find that you are going to use Warewulf, or any other cluster provisioning tool, it is very helpful to require that hardware vendors preconfigure your cluster nodes with values of your choosing, and ask them to provide a text file that includes all of the HW/MAC addresses of the compute nodes in the order they are racked (which most creditable vendors will do). You can also ask them to certify their computing stack for the operating system you wish to use and the provisioning system. This helps hardware vendors to ensure their stack works with open source projects like Warewulf, Debian, OpenSuSE, and Rocky Linux.
The Provisioning Process
========================
When the cluster node boots, the following order of operations will occur:
#. BIOS:
#. The system BIOS will bootstrap the initialization of the hardware
#. The network card will register its option ROM into the BIOS
#. The BIOS will run through all of its functions and finish with boot devices
#. The boot devices are attempted in order
#. When it gets to the network boot device, PXE is run from the firmware on the network card
#. PXE:
#. PXE will request a BOOTP/DHCP address on the network
#. The Warewulf controller's DHCP server will respond with a network configuration and filename to try and boot
#. PXE will attempt to download the filename referred to in the DHCP response via TFTP
#. The downloaded file will execute an iPXE stack which will reach out to the Warewulf server for it's configuration
#. Bootstrap:
#. The Warewulf server will generate the iPXE configuration which will include directions of what else is necessary to download and how to boot.
#. The kernel, container image, kernel modules, and system overlay are all downloaded over REST HTTP from the Warewulf Server
#. iPXE executes the kernel and processes the overlays to provide a unified root file system
#. Warewulf bootstraps the initialization of cluster node's operating system
#. File System (re)configuration
#. SELinux
#. ``wwclient`` is called as a background daemon and sleeps until network is ready
#. The Warewulf bootstrap execs the container's ``/sbin/init``
#. Container:
#. The container now boots exactly as any operating system would expect

36
userdocs/contents/security.rst Normal file
View File

@@ -0,0 +1,36 @@
========
Security
========
Historically, most HPC clusters utilize a security model that is "hard on the exterior and soft and gushy on the interior". It is not that a user has free roam once logged in, but rather we tend to rely on just simple POSIX security models on the inside. For example, one of the common practices is to completely disable SELinux on a new cluster setup. Just kill it because it gets in the way.
For that reason, most critical HPC clusters leverage VPNs and/or bastion hosts with multi-factor authentication (MFA) to help secure it on the outside. But even with MFA and secure ssh connections through a bastion host, it is still possible for malicious users to gain access to these systems. Security being like layers of an onion is accurate, but on an HPC system, those layers are predominately on the outside of the cluster, not the inside.
Warewulf was written and designed from the ground up to go a bit further. And while certain parallelization and high performance library capabilities still require lowering the security threshold, Warewulf strives to not be a blocker here.
SELinux
=======
The Warewulf server itself was developed with SELinux enabled in "targeted" and "enforcing" mode and with the firewall active.
Additionally, the provisioning process fully supports SELinux by default. In previous versions you had to enable a switch to support SELinux, but in Warewulf v4 and above, it is always enabled, but you do have to make some configuration changes.
#. The first thing to do is to change the provision "Root" option. By default this is ``initramfs`` which means, take whatever file system the kernel hands us. By default this is a ``ramfs`` type file system (however this may not always be the case) and this format does not support extended file attributes which are required for SELinux. Instead you must configure Warewulf to use ``tmpfs`` for the provisioning file system. That change is made like: ``$ sudo wwctl profile set --root tmpfs default``.
#. That is all you have to do to ensure that Warewulf will probably support SELinux. Once that is done, you just need to enable SELinux in ``/etc/sysconfig/selinux`` and install the appropriate profiles into the container.
Provisioning Security
=====================
Provisioning in generally is known to be rather "insecure" because when a user lands on a compute node, there is generally nothing stopping them from spoofing a provision request and downloading the provisioned raw materials for inspection.
In Warewulf there are two ways to secure the provisioning process:
#. The provisioning connections and transfers are not secure due to not being able to manage a secure root of trust through a PXE process. The best way to secure the provisioning process is to enact a vLAN used specifically for provisioning. Warewulf supports this but you must consult your switch documentation and features to implement a default vLAN for provisioning and ensure that the runtime operating system is configured for a different tagged vLAN once booted.
#. Warewulf will leverage hardware "asset tags" which almost all vendors support. It is a configurable string that is configured in firmware and accessible only via root or physical access. During provisioning (as well as post provisioning via ``wwclient``) Warewulf, can use the asset tag as a secure token. If you have setup your hardware with an asset tag, you simply need to tell Warewulf what that asset tag is. When the asset tag is defined in Warewulf (``wwctl node set --assetkey "..."``), it will only provision and communicate with requests from that system matching that asset tag.
Summary
=======
Warewulf does not limit the security posture of a cluster at all, and perhaps it increases it as not all provisioners work with firewalls and SELinux enabled and enforcing. But even with that, cluster security is always up to the system manager and organizational policies. Our job is just to ensure that we don't limit those policies in any way.

38
userdocs/contents/setup.rst Normal file
View File

@@ -0,0 +1,38 @@
====================
Control Server Setup
====================
Operating System Installation
=============================
Warewulf has almost no predetermined or required configurations aside from a base architecture networking layout. Install your Linux distribution of choice as you would like, but do pay attention to the cluster's private network configuration.
Network
=======
A clustered resource depends on a private management network. This network can be either persistent (it is always "up" even after provisioning) to temporary which might only be used for provisioning and/or out of band system control and management e.g. IPMI).
It is important for this management network to be private to the compute resource because Warewulf requires network services on that network which may conflict with services on the production/public network (e.g. DHCP). It is also important from a security perspective as the management network for typical HPC systems have an implied trust level associated with it and generally there is no firewalling or network monitoring occurring on these networks.
Usually, the control node is "dual homed" which means it has at least two interface cards, one connected to the private cluster network and one dedicated to the public network (as the figure above demonstrates).
> note: It is possible to omit the public network interface with a reverse NAT. Warewulf can operate in this configuration but it extends beyond the scope of this documentation.
Many clusters have more then one private network. This is common for performance critical HPC clusters that implement a high speed and low latency network like InfiniBand. In this case, this network is used for high speed data transfers for inter-process communication between compute nodes and file system IO.
Warewulf will need to be configured to use the private cluster management network. Warewulf will use this network for booting the nodes over PXE. There are three network protocols used to accomplish this DHCP/BOOT, TFTP, and HTTP on port ``9873``. Warewulf will use the operating system's provided version of DHCP (ISC-DHCP) and TFTP for the PXE bootstrap to iPXE, and then iPXE will use Warewulf's internal HTTP services to transfer the larger files for provisioning.
Addressing
==========
The addressing scheme of your private cluster network is 100% up to the system integrator, but for large clusters, many organizations like to organize the address allocations. Below is a recommended IP addressing scheme which we will use for the rest of this document.
* ``10.0.0.1``: Private network address IP
* ``255.255.252.0``: Private network subnet mask
Here is an example of how the cluster's address can be divided for a 255 node cluster:
* ``10.0.0.1 - 10.0.0.255``: Cluster infrastructure including this host, schedulers, file systems, routers, switches, etc.
* ``10.0.1.1 - 10.0.1.255``: DHCP range for booting nodes
* ``10.0.2.1 - 10.0.2.255``: Static node addresses
* ``10.0.3.1 - 10.0.3.255``: IPMI and/or out of band addresses for the compute nodes

40
userdocs/contents/stateless.rst Normal file
View File

@@ -0,0 +1,40 @@
======================
Stateless Provisioning
======================
Why is Provisioning Important
=============================
Clusters are pools of servers bundled together to do a particular job or set of jobs. While there are a number of different use cases for clustering today, Warewulf was originally designed out of necessity.
Back in 2000, when Linux clustering was growing up for HPC, the issues of scale became apparent. Of course in HPC, there are many scalability factors which needed to be overcome as we continued to scale up clusters. Pretty early on was the "administrative scaling" which is the factor that full time systems administrators could only maintain so many servers. While homogenous configurations were able to help that, we still had the problem that every installed server became a point of administration, version creep, and debugging. The larger the cluster, the harder this problem was to solve.
Warewulf was created to help with exactly this.
Provisioning Overview
=====================
Provisioning in this definition is the process of putting an operating system onto a system. There are many ways to provision operating system images, from copying hard drives, to scripted installs, to automated installs. There are many valuable tools to facilitate this and they all helped to solve this problem.
In a cluster environment, this means one could group all of the nodes together, to be installed in bulk. Previous to cluster provisioning system administrators would go around to each cluster node, and install it from scratch, with an ISO or USB thumb drive. This obviously is not scalable. But being able to automatically install hundreds or thousands of computers in parallel and automate the management of these systems completely changed the paradigm.
There were several cluster provision and management toolkits already available when Warewulf was created and while these tools absolutely helped, there was even more optimization to be had. Stateless computing.
Stateless Provisioning
======================
The next step past automated installs is to just skip the installation completely; boot directly into the runtime operating system without ever doing an installation.
This is Warewulf.
Stateless provisioning is realizing you never have to install another compute node. Think of it like booting a LiveOS or LiveISO on nodes over the network. This means that no node individually is a point of system administration, but rather the entire cluster, inclusively is administrated as a single unit.
If all cluster nodes are booting the same OS image (or set of OS images), then any individual nodes that have problems is hardware. Debugging software and doing system administration in single points within a cluster is not needed. There is no version drift, because it is not possible for nodes to fall out of sync. Every reboot makes it exactly the same as its neighbors.
Warewulf provisions the operating system by default to system memory. There is no need for hard drives with Warewulf.
Previous versions of Warewulf had the ability to write the operating system to hard disk as well as do hybrid provisioning (the core operating system in memory, other pieces overlaid over NFS) but these have been obsoleted in Warewulf v4 as there are easier ways to accomplish the same thing (e.g. use of swap space).
> note: If you wish to provision to the hard drive, we might add that feature back based on user requests but in the mean time, you may wish to look at an automated or scripted installation platform instead of a cluster provisioning system.
In our experience, the Warewulf provisioning model is by far the most advantageous, simplest, and most flexible and scalable cluster provisioning platform available.

49
userdocs/contents/templating.rst Normal file
View File

@@ -0,0 +1,49 @@
==========
Templating
==========
Warewulf uses the ``text/template`` engine to convert dynamic content into static content and auto-populate files with the appropriate data on demand.
In Warewulf, you can find templates both for the provisioning services (e.g. ``/etc/warewulf/ipxe/``, ``/etc/warewulf/dhcp/``, and ``/etc/warewulf/hosts.tmpl``) as well as within the runtime and system overlays.
(more documentation coming soon)
Examples
========
range
-----
iterate over elements of an array
.. code-block:: go
{{ range $devname, $netdev := .NetDevs }}
# netdev = {{ $netdev.Hwaddr }}
{{ end }}
increment variable in loop
^^^^^^^^^^^^^^^^^^^^^^^^^^
iterate over elements of an array and increment ``i`` each loop cycle
.. code-block:: go
{{ $i := 0 }}
{{ range $devname, $netdev := .NetDevs }}
# netdev{{$i}} = {{ $netdev.Hwaddr }}
{{ $i = inc $i }}
{{ end }}
decrement
^^^^^^^^^
iterate over elements of an array and decrement ``i`` each loop cycle
.. code-block:: go
{{ $i := 10 }}
{{ range $devname, $netdev := .NetDevs }}
# netdev{{$i}} = {{ $netdev.Hwaddr }}
{{ $i = dec $i }}
{{ end }}