kelvin/warewulf
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Move documentation

Browse Source
This commit is contained in:
Brian Clemens
2021-01-15 19:44:32 +09:00
parent 98f5142635
commit 659d652813
9 changed files with 1 additions and 524 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
docs/Beowulf-Architecture.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 91 KiB

23
docs/Contributing.md
View File

@@ -1,23 +0,0 @@
# Contributing
Warewulf is an open source project and we invite your collaboration and help.
## Contributor's Agreement
You are under no obligation whatsoever to provide any bug fixes, patches, or upgrades to the features, functionality
or performance of the source code ("Enhancements") to anyone; however, if you choose to make your Enhancements
available either publicly, or directly to the project, without imposing a separate written license agreement for such
Enhancements, then you hereby grant the following license: a non-exclusive, royalty-free perpetual license to
install, use, modify, prepare derivative works, incorporate into other computer software, distribute, and sublicense
such enhancements or derivative works thereof, in binary and source code form.
## To Contribute
Here is the very high level guide to contributing:
* For the project to a location of your choosing
* Clone your fork locally: `git clone http://github.com/$NAME/warewulf.git`
* Inside your fork, create a local branch: `cd warewulf; git checkout -b localchanges`
* Make the necessary changes and commit them: `git commit -a`
* Push your changes: `git push origin localchanges`
* Create a PR using GitHub: `https://github.com/$NAME/warewulf/pull/new/localchanges`
*note: set your environment variable `NAME=github_username` before running the above commands.*

22
docs/README.md
View File

@@ -1,22 +0,0 @@
# Documentation for Warewulf
The documentation for Warewulf is a work in progress and we are asking for contributions from the open source
community on this. Please see the [contributing](Contributing.md) document for how to help out this project.
### Quickstarts
* [RHEL-7 quickstart](rhel7-quickstart.md)
* [RHEL-8 quickstart](rhel8-quickstart.md)
### Chapters
(all coming soon)
* [Introduction](introduction.md)
* [Terms and Definitions](terms.md)
* [Architecture](architecture.md)
* [System Services and Dependencies](system_services.md)
* [Profiles](profiles.md)
* [Node Configurations](nodes.md)
* [Containers and VNFS images](containers.md)
* [Overlays](overlays.md)
* [SELinux](selinux.md)

117
docs/architecture.md
View File

@@ -1,117 +0,0 @@
# Warewulf Cluster Architecture
## System Architecture
Warewulf's primary design goal is to facilitate the typical architecture of an HPC style cluster. The roots of these
clusters originates from the foundational architecture that has influenced these architectures for the last 20-30
years, commonly known as the "Beowulf".
### The Beowulf
The original [Beowulf Cluster](https://en.wikipedia.org/wiki/Beowulf_cluster) was developed in 1996 by Dr. Thomas
Sterling and Dr. Donald Becker at Nasa. The architecture is defined as a group of similar compute worker nodes all
connected together using standard commodity equipment on a private network segment. The control node is dual homed
(has two network interface cards) with one of these network interface cards attached to the upstream network and the
other connected to the same private network which connects the compute worker nodes (as seen in the figure below).
![Beowulf System Architecture](Beowulf-Architecture.png)
This architecture is very advantageous for creating a scalable resource for HPC clustering but it is also only a
very simple portrayal of the base foundation. To this we must add storage, scheduling and resource management,
monitoring, interactive systems, and as the system grows, it maybe needed to have groups of nodes, that have
different features, architectures, vintages, memory configuration, GPUs, or interconnects. As the system grows,
the necessity for a scalable control surface also increases which can result in even more complexity.
### Provisioning
Warewulf is designed to support not only the simplest of clusters, but also the most complicated, largest, and
specialized resources that are demanded today. But in all of these configurations, we go back to this standard
architecture illustrated above. This is because Warewulf is designed to "own" the network broadcast domain due to
how these worker nodes boot.
As mentioned before, Warewulf is designed first and foremost to be a stateless provisioning subsystem. This means
that the worker nodes maintain no state about their configuration, operating system, or other when they are powered
off. The result of this is when these systems boot, they retain no knowledge on who they are, what they are supposed
to do, or how they got there. For these reasons, these systems will need to boot via
[PXE](https://en.wikipedia.org/wiki/Preboot_Execution_Environment).
#### PXE
By Default, most servers will have network interface cards that support PXE. In a nutshell, PXE will allow the
network card to be seen by the BIOS as a bootable device. This means that the boot order may need to be configured
in the system's BIOS on the worker nodes to allow it to boot. If there is also another bootable device on these
systems, it might be necessary to set the network interface card to boot first.
When the system boots via PXE, it will begin a chain reaction of events:
1. The network card will register an option rom into the BIOS
1. The BIOS will run through all of its functions and finishes with boot devices
1. The boot devices are attempted to be booted in the defined order
1. When it gets to the network boot device, PXE is run from the firmware on the network card
1. PXE will request a Bootp/DHCP on the network
1. If the DHCP response includes a boot file name, it will download this file (iPXE boot image) over TFTP
1. Once iPXE is downloaded, it is loaded by the network card and it will request a configuration from the
server
1. The configuration will tell iPXE what to download and load
Warewulf manages the entire process once the worker node's network device has begun the PXE process.
#### Warewulf Server
Warewulf will configure and setup the controller's DHCP and TFTP services, as well as putting all the required
files into place but once the iPXE file has been sent and loaded on the nodes, all network communication from this
point on is being handled by the Warewulf server over HTTP. The order of events from this point on are as follows:
1. iPXE requests it's configuration and Warewulf generates this on demand from the configured template
1. The default iPXE template tells iPXE to request a kernel, VNFS image, runtime kernel modules, and a system overlay
1. Each of the requested files will be "served" to the node
#### Post PXE
Once the worker node has received all the required files, the kernel will be booted and the runtime components will be
loaded into the kernel's initial ram file system (initramfs). The order of this is very because each layer is
"layered" on top of the previous. The layers are implemented in the following order:
1. VNFS/Container image
1. Kernel modules
1. System Overlay
1. Runtime Overlay (repeated at a given interval)
As part of the provisioning process, the system boots after the System Overlay has been provisioned which means that
the Runtime Overlay occurs after `/sbin/init` has been called and the system. This delineation is important because
it clearly defines what should be in the system Vs. runtime overlay. More on overlays in the next section.
Before calling `/sbin/init` (or `init` to override in the node configuration file), Warewulf must set up the system.
Again, this process occurs after the System Overlay has been provisioned. This is where the system initializes and
prepares for booting the runtime OS. Depending on the configuration, the system might boot directly in the initramfs
file system as it stands, or it could migrate the root file system to a different mount point (e.g. `tmpfs` or at
some point, hard drives).
Other things that get done at this stage are setting up and enabling SELinux, IPMI, making any needed changes or
configurations to the file system before booting it, and starting up the `wwclient` which is responsible for loading
the runtime overlay.
Lastly, the `init` process is executed with PID 1 and thus "boots" the VNFS container. This is where SysVInit,
Upstart, or Systemd takes over.
#### Warewulf Overlays
As described above, Warewulf uses layers to provision. The first layer is static across any number of nodes, but
each node may require some custom configurations, for example, network. This means there must be a method for
leveraging a base file system which can be shared by many nodes and also be able to configure these file systems
with custom per-node options at a large scale.
Typically, there are two major times this configuration needs to be done. Pre "boot" and post "boot". In this case, as
described above, we can consider the call to `/sbin/init` the delineation point and the proper way to consider the two
user configurable overlays: system and runtime.
**System Overlay**: The system overlay is what will be present before `/sbin/init` is called. This gives the
administrator the ability to control the configuration of the booting system itself. For example, network
configuration is necessary to be configured on every node, but each node must have a different network configuration
otherwise the IP addresses will clash and this must be set before Systemd brings up the network device. The Warewulf
system overlay is the right place to configure this.
**Runtime Overlays**: Some configurations happen after the system boots and continuously at periodic intervals. For
example user and group accounts. You probably don't want to reprovision a node to add a user or change a runtime
configuration, and this is where you will need to use the runtime overlay.
Both overlays leverage a similar file system template structure. Each overlay (you can create any number of them)
can include text files, directories, links, and templates. Templates allow you to customize any of the content
within an overlay dynamically for each node that will be leveraging that template.

45
docs/introduction.md
View File

@@ -1,45 +0,0 @@
# Introduction
Warewulf is an operating system provisioning system for Linux. It was created in 2001 to facilitate and scale the
administration of HPC clusters in a way that allowed for simple, turn key solutions as well as support flexibility
and configurability at scale.
Since its initial release, Warewulf has become the most popular open source and vendor-agnostic provisioning system
within the HPC ecosystem with worldwide adoption. Warewulf is known for its massive scalability and ease of use for
stateless (disk optional) provisioning.
In a nutshell, node operating system images are containers. These containers are built into a bootable format called
a "Virtual Node File System" (VNFS) image which is provisioned out to nodes at boot. The VNFS and kernel can be
configured for any number of nodes which means you could have all of your nodes booting the exact same container
VNFS image. The subtle differences in operating system configurations between the nodes are handled in "overlays".
On boot, each node receives, in the following order:
1. iPXE boot image (tftp)
1. Linux kernel (http)
1. VNFS image (http)
1. Kernel module overlay (http)
1. System overlay (http)
*note: once the node has booted, it will request a runtime overlay (privileged http) at a periodic interval.*
All the images which are transferred for provisioning are prebuilt static components and the per node connection
processing is minimal. This means that Warewulf is as scalable as the physical infrastructure.
### Warewulf Design Tenants
To enable simple, scalable, and flexible operating system management at scale.
1. **Lightweight**: Warewulf needs to do it's job and stay out of the way. There should be no underlying system
dependencies, changes, or "stack" for the controller or worker nodes.
1. **Simple**: Warewulf is used by hobbyists, researchers, scientists, as well as systems admnistrators and engineers.
This means that Warewulf must be simple to use and intuitive to understand.
1. **Flexible**: Warewulf must remain highly flexible to be able to fit in any environment, from a computer lab with
graphical workstations, to an under-desk clusters, as well as massive centers providing traditional HPC
capabilities to thousands of users.
1. **Agnostic**: From the underlying Linux distribution to the underlying hardware, Warewulf should be as agnostic
and standards compliant as possible. From ARM to X86, Atos to Dell, Warewulf can provision it all equally well
with no favorites.
1. **Open Source**: It is imperative that Warewulf be and remain absolutely Open Source.

146
docs/rhel7-quickstart.md
View File

@@ -1,146 +0,0 @@
## Quick Start for RHEL, CentOS, and Rocky Linux version 7
#### Install Warewulf and dependencies
```
sudo yum install epel-release
sudo yum install golang tftp-server dhcp nfs-utils
sudo systemctl stop firewalld
sudo systemctl disable firewalld
git clone https://github.com/ctrliq/warewulf.git
cd warewulf
make all
sudo make install
```
#### Configure the controller:
Edit the file `/etc/warewulf/warewulf.conf` and ensure that you've set the
appropriate configuration paramaters. Here are some of the defaults for reference:
```
ipaddr: 192.168.1.1
netmask: 255.255.255.0
warewulf:
port: 9873
secure: true
update interval: 60
dhcp:
enabled: true
range start: 192.168.1.150
range end: 192.168.1.200
template: default
systemd name: dhcpd
tftp:
enabled: true
tftproot: /var/lib/tftpboot
systemd name: tftp
nfs:
systemd name: nfs-server
exports:
- /home
- /var/warewulf
```
#### Configure system services automatically
```
sudo ./wwctl configure dhcp # Create the default dhcpd.conf file and start/enable service
sudo ./wwctl configure tftp # Install the base tftp/PXE boot files and start/enable service
sudo ./wwctl configure nfs # Configure the exports and create an fstab in the default system overlay
sudo ./wwctl configure ssh # Build the basic ssh keys to be included by the default system overlay
```
#### Pull and build the VNFS container and kernel:
This will pull a basic VNFS container from Docker Hub and import the default running
kernel from the controller node and set both in the "default" node profile.
```
sudo ./wwctl container pull docker://warewulf/centos-7 centos-7 --setdefault
sudo ./wwctl kernel build $(uname -r) --setdefault
```
#### Set up the default node profile
The `--setdefault` arguments above will automatically set those entries in the default
profile, but if you wanted to set them by hand to something different, you can do the
following:
```
sudo ./wwctl profile set default -K $(uname -r) -C centos-7
```
Next we set some default networking configurations for the first ethernet device. On
modern Linux distributions, the name of the device is not critical, as it will be setup
according to the HW address. Because all nodes will share the netmask and gateway
configuration, we can set them in the default profile as follows:
```
sudo ./wwctl profile set default --netdev eth0 -M 255.255.255.0 -G 192.168.1.1
sudo ./wwctl profile list
```
#### Add a node and build node specific overlays
Adding nodes can be done while setting configurations in one command. Here we are setting
the IP address of `eth0` and setting this node to be discoverable, which will then
automatically have the HW address added to the configuration as the node boots.
Node names must be unique. If you have node groups and/or multiple clusters, designate
them using dot notation.
Note that the full node configuration comes from both cascading profiles and node
configurations which always supersede profile configurations.
```
sudo ./wwctl node add n0000.cluster --netdev eth0 -I 192.168.1.100 --discoverable
sudo ./wwctl node list -a n0000
```
### Warewulf Overlays
There are two types of overlays: system and runtime overlays.
System overlays are provisioned to the node before `/sbin/init` is called. This enables us
to prepopulate node configurations with content that is node specific like networking and
service configurations.
Runtime overlays are provisioned after the node has booted and periodically during the
normal runtime of the node. Because these overlays are provisioned at periodic intervals,
they are very useful for content that changes, like users and groups.
Overlays are generated from a template structure that is viewed using the `wwctl overlay`
commands. Files that end in the `.ww` suffix are templates and abide by standard
text/template rules. This supports loops, arrays, variables, and functions making overlays
extremely flexible.
*note: When using the overlay subsystem, system overlays are never shown by default. So
when running `overlay` commands, you are always looking at runtime overlays unless the
`-s` option is passed.*
All overlays are compiled before being provisioned. This accelerates the provisioning
process because there is less to do when nodes are being managed at scale.
Here are some of the common `overlay` commands:
```
sudo ./wwctl overlay list -l
sudo ./wwctl overlay list -ls
sudo ./wwctl overlay edit default /etc/hello_world.ww
sudo ./wwctl overlay build -a
```
#### Start the Warewulf daemon:
Once the above provisioning images are built, you can check the provisioning "rediness"
and then begin booting nodes.
```
sudo ./wwctl ready
sudo ./wwctl server start
sudo ./wwctl server status
```
#### Boot your compute node and watch it boot

147
docs/rhel8-quickstart.md
View File

@@ -1,147 +0,0 @@
## Quick Start for RHEL, CentOS, and Rocky Linux version 8
#### Install Warewulf and dependencies
```
sudo yum groupinstall "Development Tools"
sudo yum install epel-release
sudo yum install golang tftp-server dhcp-server nfs-utils
sudo systemctl stop firewalld
sudo systemctl disable firewalld
git clone https://github.com/ctrliq/warewulf.git
cd warewulf
make all
sudo make install
```
#### Configure the controller:
Edit the file `/etc/warewulf/warewulf.conf` and ensure that you've set the
appropriate configuration paramaters. Here are some of the defaults for reference:
```
ipaddr: 192.168.1.1
netmask: 255.255.255.0
warewulf:
port: 9873
secure: true
update interval: 60
dhcp:
enabled: true
range start: 192.168.1.150
range end: 192.168.1.200
template: default
systemd name: dhcpd
tftp:
enabled: true
tftproot: /var/lib/tftpboot
systemd name: tftp
nfs:
systemd name: nfs-server
exports:
- /home
- /var/warewulf
```
#### Configure system services automatically
```
sudo ./wwctl configure dhcp # Create the default dhcpd.conf file and start/enable service
sudo ./wwctl configure tftp # Install the base tftp/PXE boot files and start/enable service
sudo ./wwctl configure nfs # Configure the exports and create an fstab in the default system overlay
sudo ./wwctl configure ssh # Build the basic ssh keys to be included by the default system overlay
```
#### Pull and build the VNFS container and kernel:
This will pull a basic VNFS container from Docker Hub and import the default running
kernel from the controller node and set both in the "default" node profile.
```
sudo ./wwctl container pull docker://warewulf/centos-8 centos-8 --setdefault
sudo ./wwctl kernel build $(uname -r) --setdefault
```
#### Set up the default node profile
The `--setdefault` arguments above will automatically set those entries in the default
profile, but if you wanted to set them by hand to something different, you can do the
following:
```
sudo ./wwctl profile set default -K $(uname -r) -C centos-7
```
Next we set some default networking configurations for the first ethernet device. On
modern Linux distributions, the name of the device is not critical, as it will be setup
according to the HW address. Because all nodes will share the netmask and gateway
configuration, we can set them in the default profile as follows:
```
sudo ./wwctl profile set default --netdev eth0 -M 255.255.255.0 -G 192.168.1.1
sudo ./wwctl profile list
```
#### Add a node and build node specific overlays
Adding nodes can be done while setting configurations in one command. Here we are setting
the IP address of `eth0` and setting this node to be discoverable, which will then
automatically have the HW address added to the configuration as the node boots.
Node names must be unique. If you have node groups and/or multiple clusters, designate
them using dot notation.
Note that the full node configuration comes from both cascading profiles and node
configurations which always supersede profile configurations.
```
sudo ./wwctl node add n0000.cluster --netdev eth0 -I 192.168.1.100 --discoverable
sudo ./wwctl node list -a n0000
```
### Warewulf Overlays
There are two types of overlays: system and runtime overlays.
System overlays are provisioned to the node before `/sbin/init` is called. This enables us
to prepopulate node configurations with content that is node specific like networking and
service configurations.
Runtime overlays are provisioned after the node has booted and periodically during the
normal runtime of the node. Because these overlays are provisioned at periodic intervals,
they are very useful for content that changes, like users and groups.
Overlays are generated from a template structure that is viewed using the `wwctl overlay`
commands. Files that end in the `.ww` suffix are templates and abide by standard
text/template rules. This supports loops, arrays, variables, and functions making overlays
extremely flexible.
*note: When using the overlay subsystem, system overlays are never shown by default. So
when running `overlay` commands, you are always looking at runtime overlays unless the
`-s` option is passed.*
All overlays are compiled before being provisioned. This accelerates the provisioning
process because there is less to do when nodes are being managed at scale.
Here are some of the common `overlay` commands:
```
sudo ./wwctl overlay list -l
sudo ./wwctl overlay list -ls
sudo ./wwctl overlay edit default /etc/hello_world.ww
sudo ./wwctl overlay build -a
```
#### Start the Warewulf daemon:
Once the above provisioning images are built, you can check the provisioning "rediness"
and then begin booting nodes.
```
sudo ./wwctl ready
sudo ./wwctl server start
sudo ./wwctl server status
```
#### Boot your compute node and watch it boot

23
docs/terms.md
View File

@@ -1,23 +0,0 @@
# Terms and Definitions
### Controller
The controller node(s) are the resources responsible for management, control, and administration of the cluster.
Historically these systems have been called "master", "head", or "administrative" nodes, but we feel the term
"controller" is more appropriate and descriptive of the role of this system.
### Workers
Worker nodes are the systems that are being provisioned by Warewulf. The roles of these systems could be "compute",
"storage", "GPU", "IO", etc. which would typically be used as a prefix, for example: "**compute worker node**"
### Container
Containers are used by Warewulf as the template for the VNFS image. Warewulf containers can be any type of OCI or
Singularity standard image formats but maintained on disk as an "OCI bundle". Warewulf integrates with Docker,
Docker Hub, any OCI registery, Singularity, standard chroots, etc.
### Virtual Node File System (VNFS)
### Overlays
### Kernel
### Initramfs