diff --git a/README.md b/README.md index a070b086..98ba8a9b 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ #### Quick Links: -* [Documentation](docs/README.md) +* [Documentation](https://warewulf.readthedocs.io) * [GitHub](http://github.com/ctrliq/warewulf) ## About Warewulf diff --git a/docs/Beowulf-Architecture.png b/docs/Beowulf-Architecture.png deleted file mode 100644 index 8c75d919..00000000 Binary files a/docs/Beowulf-Architecture.png and /dev/null differ diff --git a/docs/Contributing.md b/docs/Contributing.md deleted file mode 100644 index 29bd092e..00000000 --- a/docs/Contributing.md +++ /dev/null @@ -1,23 +0,0 @@ -# Contributing - -Warewulf is an open source project and we invite your collaboration and help. - -## Contributor's Agreement -You are under no obligation whatsoever to provide any bug fixes, patches, or upgrades to the features, functionality -or performance of the source code ("Enhancements") to anyone; however, if you choose to make your Enhancements -available either publicly, or directly to the project, without imposing a separate written license agreement for such -Enhancements, then you hereby grant the following license: a non-exclusive, royalty-free perpetual license to -install, use, modify, prepare derivative works, incorporate into other computer software, distribute, and sublicense -such enhancements or derivative works thereof, in binary and source code form. - -## To Contribute -Here is the very high level guide to contributing: - -* For the project to a location of your choosing -* Clone your fork locally: `git clone http://github.com/$NAME/warewulf.git` -* Inside your fork, create a local branch: `cd warewulf; git checkout -b localchanges` -* Make the necessary changes and commit them: `git commit -a` -* Push your changes: `git push origin localchanges` -* Create a PR using GitHub: `https://github.com/$NAME/warewulf/pull/new/localchanges` - -*note: set your environment variable `NAME=github_username` before running the above commands.* diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index d011d1ee..00000000 --- a/docs/README.md +++ /dev/null @@ -1,22 +0,0 @@ -# Documentation for Warewulf - -The documentation for Warewulf is a work in progress and we are asking for contributions from the open source -community on this. Please see the [contributing](Contributing.md) document for how to help out this project. - -### Quickstarts - -* [RHEL-7 quickstart](rhel7-quickstart.md) -* [RHEL-8 quickstart](rhel8-quickstart.md) - -### Chapters -(all coming soon) - -* [Introduction](introduction.md) -* [Terms and Definitions](terms.md) -* [Architecture](architecture.md) -* [System Services and Dependencies](system_services.md) -* [Profiles](profiles.md) -* [Node Configurations](nodes.md) -* [Containers and VNFS images](containers.md) -* [Overlays](overlays.md) -* [SELinux](selinux.md) diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index 8462357e..00000000 --- a/docs/architecture.md +++ /dev/null @@ -1,117 +0,0 @@ -# Warewulf Cluster Architecture - -## System Architecture -Warewulf's primary design goal is to facilitate the typical architecture of an HPC style cluster. The roots of these -clusters originates from the foundational architecture that has influenced these architectures for the last 20-30 -years, commonly known as the "Beowulf". - -### The Beowulf -The original [Beowulf Cluster](https://en.wikipedia.org/wiki/Beowulf_cluster) was developed in 1996 by Dr. Thomas -Sterling and Dr. Donald Becker at Nasa. The architecture is defined as a group of similar compute worker nodes all -connected together using standard commodity equipment on a private network segment. The control node is dual homed -(has two network interface cards) with one of these network interface cards attached to the upstream network and the -other connected to the same private network which connects the compute worker nodes (as seen in the figure below). - -![Beowulf System Architecture](Beowulf-Architecture.png) - -This architecture is very advantageous for creating a scalable resource for HPC clustering but it is also only a -very simple portrayal of the base foundation. To this we must add storage, scheduling and resource management, -monitoring, interactive systems, and as the system grows, it maybe needed to have groups of nodes, that have -different features, architectures, vintages, memory configuration, GPUs, or interconnects. As the system grows, -the necessity for a scalable control surface also increases which can result in even more complexity. - -### Provisioning -Warewulf is designed to support not only the simplest of clusters, but also the most complicated, largest, and -specialized resources that are demanded today. But in all of these configurations, we go back to this standard -architecture illustrated above. This is because Warewulf is designed to "own" the network broadcast domain due to -how these worker nodes boot. - -As mentioned before, Warewulf is designed first and foremost to be a stateless provisioning subsystem. This means -that the worker nodes maintain no state about their configuration, operating system, or other when they are powered -off. The result of this is when these systems boot, they retain no knowledge on who they are, what they are supposed -to do, or how they got there. For these reasons, these systems will need to boot via -[PXE](https://en.wikipedia.org/wiki/Preboot_Execution_Environment). - -#### PXE -By Default, most servers will have network interface cards that support PXE. In a nutshell, PXE will allow the -network card to be seen by the BIOS as a bootable device. This means that the boot order may need to be configured -in the system's BIOS on the worker nodes to allow it to boot. If there is also another bootable device on these -systems, it might be necessary to set the network interface card to boot first. - -When the system boots via PXE, it will begin a chain reaction of events: - -1. The network card will register an option rom into the BIOS -1. The BIOS will run through all of its functions and finishes with boot devices -1. The boot devices are attempted to be booted in the defined order -1. When it gets to the network boot device, PXE is run from the firmware on the network card -1. PXE will request a Bootp/DHCP on the network -1. If the DHCP response includes a boot file name, it will download this file (iPXE boot image) over TFTP -1. Once iPXE is downloaded, it is loaded by the network card and it will request a configuration from the - server -1. The configuration will tell iPXE what to download and load - -Warewulf manages the entire process once the worker node's network device has begun the PXE process. - -#### Warewulf Server -Warewulf will configure and setup the controller's DHCP and TFTP services, as well as putting all the required -files into place but once the iPXE file has been sent and loaded on the nodes, all network communication from this -point on is being handled by the Warewulf server over HTTP. The order of events from this point on are as follows: - -1. iPXE requests it's configuration and Warewulf generates this on demand from the configured template -1. The default iPXE template tells iPXE to request a kernel, VNFS image, runtime kernel modules, and a system overlay -1. Each of the requested files will be "served" to the node - -#### Post PXE -Once the worker node has received all the required files, the kernel will be booted and the runtime components will be -loaded into the kernel's initial ram file system (initramfs). The order of this is very because each layer is -"layered" on top of the previous. The layers are implemented in the following order: - -1. VNFS/Container image -1. Kernel modules -1. System Overlay -1. Runtime Overlay (repeated at a given interval) - -As part of the provisioning process, the system boots after the System Overlay has been provisioned which means that -the Runtime Overlay occurs after `/sbin/init` has been called and the system. This delineation is important because -it clearly defines what should be in the system Vs. runtime overlay. More on overlays in the next section. - -Before calling `/sbin/init` (or `init` to override in the node configuration file), Warewulf must set up the system. -Again, this process occurs after the System Overlay has been provisioned. This is where the system initializes and -prepares for booting the runtime OS. Depending on the configuration, the system might boot directly in the initramfs -file system as it stands, or it could migrate the root file system to a different mount point (e.g. `tmpfs` or at -some point, hard drives). - -Other things that get done at this stage are setting up and enabling SELinux, IPMI, making any needed changes or -configurations to the file system before booting it, and starting up the `wwclient` which is responsible for loading -the runtime overlay. - -Lastly, the `init` process is executed with PID 1 and thus "boots" the VNFS container. This is where SysVInit, -Upstart, or Systemd takes over. - -#### Warewulf Overlays -As described above, Warewulf uses layers to provision. The first layer is static across any number of nodes, but -each node may require some custom configurations, for example, network. This means there must be a method for -leveraging a base file system which can be shared by many nodes and also be able to configure these file systems -with custom per-node options at a large scale. - -Typically, there are two major times this configuration needs to be done. Pre "boot" and post "boot". In this case, as -described above, we can consider the call to `/sbin/init` the delineation point and the proper way to consider the two -user configurable overlays: system and runtime. - -**System Overlay**: The system overlay is what will be present before `/sbin/init` is called. This gives the -administrator the ability to control the configuration of the booting system itself. For example, network -configuration is necessary to be configured on every node, but each node must have a different network configuration -otherwise the IP addresses will clash and this must be set before Systemd brings up the network device. The Warewulf -system overlay is the right place to configure this. - -**Runtime Overlays**: Some configurations happen after the system boots and continuously at periodic intervals. For -example user and group accounts. You probably don't want to reprovision a node to add a user or change a runtime -configuration, and this is where you will need to use the runtime overlay. - -Both overlays leverage a similar file system template structure. Each overlay (you can create any number of them) -can include text files, directories, links, and templates. Templates allow you to customize any of the content -within an overlay dynamically for each node that will be leveraging that template. - - - - diff --git a/docs/introduction.md b/docs/introduction.md deleted file mode 100644 index d8ba4a9c..00000000 --- a/docs/introduction.md +++ /dev/null @@ -1,45 +0,0 @@ -# Introduction -Warewulf is an operating system provisioning system for Linux. It was created in 2001 to facilitate and scale the -administration of HPC clusters in a way that allowed for simple, turn key solutions as well as support flexibility -and configurability at scale. - -Since its initial release, Warewulf has become the most popular open source and vendor-agnostic provisioning system -within the HPC ecosystem with worldwide adoption. Warewulf is known for its massive scalability and ease of use for -stateless (disk optional) provisioning. - -In a nutshell, node operating system images are containers. These containers are built into a bootable format called -a "Virtual Node File System" (VNFS) image which is provisioned out to nodes at boot. The VNFS and kernel can be -configured for any number of nodes which means you could have all of your nodes booting the exact same container -VNFS image. The subtle differences in operating system configurations between the nodes are handled in "overlays". - -On boot, each node receives, in the following order: - -1. iPXE boot image (tftp) -1. Linux kernel (http) -1. VNFS image (http) -1. Kernel module overlay (http) -1. System overlay (http) - -*note: once the node has booted, it will request a runtime overlay (privileged http) at a periodic interval.* - -All the images which are transferred for provisioning are prebuilt static components and the per node connection -processing is minimal. This means that Warewulf is as scalable as the physical infrastructure. - -### Warewulf Design Tenants -To enable simple, scalable, and flexible operating system management at scale. - -1. **Lightweight**: Warewulf needs to do it's job and stay out of the way. There should be no underlying system - dependencies, changes, or "stack" for the controller or worker nodes. - -1. **Simple**: Warewulf is used by hobbyists, researchers, scientists, as well as systems admnistrators and engineers. - This means that Warewulf must be simple to use and intuitive to understand. - -1. **Flexible**: Warewulf must remain highly flexible to be able to fit in any environment, from a computer lab with - graphical workstations, to an under-desk clusters, as well as massive centers providing traditional HPC - capabilities to thousands of users. - -1. **Agnostic**: From the underlying Linux distribution to the underlying hardware, Warewulf should be as agnostic - and standards compliant as possible. From ARM to X86, Atos to Dell, Warewulf can provision it all equally well - with no favorites. - -1. **Open Source**: It is imperative that Warewulf be and remain absolutely Open Source. diff --git a/docs/rhel7-quickstart.md b/docs/rhel7-quickstart.md deleted file mode 100644 index aaf02d5e..00000000 --- a/docs/rhel7-quickstart.md +++ /dev/null @@ -1,146 +0,0 @@ -## Quick Start for RHEL, CentOS, and Rocky Linux version 7 - -#### Install Warewulf and dependencies -``` -sudo yum install epel-release -sudo yum install golang tftp-server dhcp nfs-utils - -sudo systemctl stop firewalld -sudo systemctl disable firewalld - -git clone https://github.com/ctrliq/warewulf.git -cd warewulf -make all -sudo make install -``` - -#### Configure the controller: - -Edit the file `/etc/warewulf/warewulf.conf` and ensure that you've set the -appropriate configuration paramaters. Here are some of the defaults for reference: - -``` -ipaddr: 192.168.1.1 -netmask: 255.255.255.0 -warewulf: - port: 9873 - secure: true - update interval: 60 -dhcp: - enabled: true - range start: 192.168.1.150 - range end: 192.168.1.200 - template: default - systemd name: dhcpd -tftp: - enabled: true - tftproot: /var/lib/tftpboot - systemd name: tftp -nfs: - systemd name: nfs-server - exports: - - /home - - /var/warewulf -``` - -#### Configure system services automatically - -``` -sudo ./wwctl configure dhcp # Create the default dhcpd.conf file and start/enable service -sudo ./wwctl configure tftp # Install the base tftp/PXE boot files and start/enable service -sudo ./wwctl configure nfs # Configure the exports and create an fstab in the default system overlay -sudo ./wwctl configure ssh # Build the basic ssh keys to be included by the default system overlay -``` - - -#### Pull and build the VNFS container and kernel: -This will pull a basic VNFS container from Docker Hub and import the default running -kernel from the controller node and set both in the "default" node profile. - -``` -sudo ./wwctl container pull docker://warewulf/centos-7 centos-7 --setdefault -sudo ./wwctl kernel build $(uname -r) --setdefault -``` - -#### Set up the default node profile - -The `--setdefault` arguments above will automatically set those entries in the default -profile, but if you wanted to set them by hand to something different, you can do the -following: - -``` -sudo ./wwctl profile set default -K $(uname -r) -C centos-7 -``` - -Next we set some default networking configurations for the first ethernet device. On -modern Linux distributions, the name of the device is not critical, as it will be setup -according to the HW address. Because all nodes will share the netmask and gateway -configuration, we can set them in the default profile as follows: - -``` -sudo ./wwctl profile set default --netdev eth0 -M 255.255.255.0 -G 192.168.1.1 -sudo ./wwctl profile list -``` - -#### Add a node and build node specific overlays - -Adding nodes can be done while setting configurations in one command. Here we are setting -the IP address of `eth0` and setting this node to be discoverable, which will then -automatically have the HW address added to the configuration as the node boots. - -Node names must be unique. If you have node groups and/or multiple clusters, designate -them using dot notation. - -Note that the full node configuration comes from both cascading profiles and node -configurations which always supersede profile configurations. - -``` -sudo ./wwctl node add n0000.cluster --netdev eth0 -I 192.168.1.100 --discoverable -sudo ./wwctl node list -a n0000 -``` - -### Warewulf Overlays - -There are two types of overlays: system and runtime overlays. - -System overlays are provisioned to the node before `/sbin/init` is called. This enables us -to prepopulate node configurations with content that is node specific like networking and -service configurations. - -Runtime overlays are provisioned after the node has booted and periodically during the -normal runtime of the node. Because these overlays are provisioned at periodic intervals, -they are very useful for content that changes, like users and groups. - -Overlays are generated from a template structure that is viewed using the `wwctl overlay` -commands. Files that end in the `.ww` suffix are templates and abide by standard -text/template rules. This supports loops, arrays, variables, and functions making overlays -extremely flexible. - -*note: When using the overlay subsystem, system overlays are never shown by default. So -when running `overlay` commands, you are always looking at runtime overlays unless the -`-s` option is passed.* - -All overlays are compiled before being provisioned. This accelerates the provisioning -process because there is less to do when nodes are being managed at scale. - -Here are some of the common `overlay` commands: - -``` -sudo ./wwctl overlay list -l -sudo ./wwctl overlay list -ls -sudo ./wwctl overlay edit default /etc/hello_world.ww -sudo ./wwctl overlay build -a -``` - -#### Start the Warewulf daemon: - -Once the above provisioning images are built, you can check the provisioning "rediness" -and then begin booting nodes. - -``` -sudo ./wwctl ready -sudo ./wwctl server start -sudo ./wwctl server status -``` - -#### Boot your compute node and watch it boot diff --git a/docs/rhel8-quickstart.md b/docs/rhel8-quickstart.md deleted file mode 100644 index 1b255256..00000000 --- a/docs/rhel8-quickstart.md +++ /dev/null @@ -1,147 +0,0 @@ -## Quick Start for RHEL, CentOS, and Rocky Linux version 8 - -#### Install Warewulf and dependencies -``` -sudo yum groupinstall "Development Tools" -sudo yum install epel-release -sudo yum install golang tftp-server dhcp-server nfs-utils - -sudo systemctl stop firewalld -sudo systemctl disable firewalld - -git clone https://github.com/ctrliq/warewulf.git -cd warewulf -make all -sudo make install -``` - -#### Configure the controller: - -Edit the file `/etc/warewulf/warewulf.conf` and ensure that you've set the -appropriate configuration paramaters. Here are some of the defaults for reference: - -``` -ipaddr: 192.168.1.1 -netmask: 255.255.255.0 -warewulf: - port: 9873 - secure: true - update interval: 60 -dhcp: - enabled: true - range start: 192.168.1.150 - range end: 192.168.1.200 - template: default - systemd name: dhcpd -tftp: - enabled: true - tftproot: /var/lib/tftpboot - systemd name: tftp -nfs: - systemd name: nfs-server - exports: - - /home - - /var/warewulf -``` - -#### Configure system services automatically - -``` -sudo ./wwctl configure dhcp # Create the default dhcpd.conf file and start/enable service -sudo ./wwctl configure tftp # Install the base tftp/PXE boot files and start/enable service -sudo ./wwctl configure nfs # Configure the exports and create an fstab in the default system overlay -sudo ./wwctl configure ssh # Build the basic ssh keys to be included by the default system overlay -``` - - -#### Pull and build the VNFS container and kernel: -This will pull a basic VNFS container from Docker Hub and import the default running -kernel from the controller node and set both in the "default" node profile. - -``` -sudo ./wwctl container pull docker://warewulf/centos-8 centos-8 --setdefault -sudo ./wwctl kernel build $(uname -r) --setdefault -``` - -#### Set up the default node profile - -The `--setdefault` arguments above will automatically set those entries in the default -profile, but if you wanted to set them by hand to something different, you can do the -following: - -``` -sudo ./wwctl profile set default -K $(uname -r) -C centos-7 -``` - -Next we set some default networking configurations for the first ethernet device. On -modern Linux distributions, the name of the device is not critical, as it will be setup -according to the HW address. Because all nodes will share the netmask and gateway -configuration, we can set them in the default profile as follows: - -``` -sudo ./wwctl profile set default --netdev eth0 -M 255.255.255.0 -G 192.168.1.1 -sudo ./wwctl profile list -``` - -#### Add a node and build node specific overlays - -Adding nodes can be done while setting configurations in one command. Here we are setting -the IP address of `eth0` and setting this node to be discoverable, which will then -automatically have the HW address added to the configuration as the node boots. - -Node names must be unique. If you have node groups and/or multiple clusters, designate -them using dot notation. - -Note that the full node configuration comes from both cascading profiles and node -configurations which always supersede profile configurations. - -``` -sudo ./wwctl node add n0000.cluster --netdev eth0 -I 192.168.1.100 --discoverable -sudo ./wwctl node list -a n0000 -``` - -### Warewulf Overlays - -There are two types of overlays: system and runtime overlays. - -System overlays are provisioned to the node before `/sbin/init` is called. This enables us -to prepopulate node configurations with content that is node specific like networking and -service configurations. - -Runtime overlays are provisioned after the node has booted and periodically during the -normal runtime of the node. Because these overlays are provisioned at periodic intervals, -they are very useful for content that changes, like users and groups. - -Overlays are generated from a template structure that is viewed using the `wwctl overlay` -commands. Files that end in the `.ww` suffix are templates and abide by standard -text/template rules. This supports loops, arrays, variables, and functions making overlays -extremely flexible. - -*note: When using the overlay subsystem, system overlays are never shown by default. So -when running `overlay` commands, you are always looking at runtime overlays unless the -`-s` option is passed.* - -All overlays are compiled before being provisioned. This accelerates the provisioning -process because there is less to do when nodes are being managed at scale. - -Here are some of the common `overlay` commands: - -``` -sudo ./wwctl overlay list -l -sudo ./wwctl overlay list -ls -sudo ./wwctl overlay edit default /etc/hello_world.ww -sudo ./wwctl overlay build -a -``` - -#### Start the Warewulf daemon: - -Once the above provisioning images are built, you can check the provisioning "rediness" -and then begin booting nodes. - -``` -sudo ./wwctl ready -sudo ./wwctl server start -sudo ./wwctl server status -``` - -#### Boot your compute node and watch it boot diff --git a/docs/terms.md b/docs/terms.md deleted file mode 100644 index 1fbcb2d5..00000000 --- a/docs/terms.md +++ /dev/null @@ -1,23 +0,0 @@ -# Terms and Definitions - -### Controller -The controller node(s) are the resources responsible for management, control, and administration of the cluster. -Historically these systems have been called "master", "head", or "administrative" nodes, but we feel the term -"controller" is more appropriate and descriptive of the role of this system. - -### Workers -Worker nodes are the systems that are being provisioned by Warewulf. The roles of these systems could be "compute", -"storage", "GPU", "IO", etc. which would typically be used as a prefix, for example: "**compute worker node**" - -### Container -Containers are used by Warewulf as the template for the VNFS image. Warewulf containers can be any type of OCI or -Singularity standard image formats but maintained on disk as an "OCI bundle". Warewulf integrates with Docker, -Docker Hub, any OCI registery, Singularity, standard chroots, etc. - -### Virtual Node File System (VNFS) - -### Overlays - -### Kernel - -### Initramfs \ No newline at end of file