kelvin/warewulf
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Document configuring the arp cache for large clusters

Browse Source 
Signed-off-by: Jonathon Anderson <janderson@ciq.com>
This commit is contained in:
Jonathon Anderson
2026-03-11 23:22:07 -06:00
parent fe8f4b5b11
commit 3f12d5635c
2 changed files with 56 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
userdocs/server/routes.rst
View File

@@ -181,6 +181,12 @@ of the client's IP address against the kernel's ARP cache (``/proc/net/arp``).
This route is intended for EFI HTTP Boot clients, where the firmware fetches This route is intended for EFI HTTP Boot clients, where the firmware fetches
a boot URI from DHCP and cannot perform variable substitution. a boot URI from DHCP and cannot perform variable substitution.
.. note::
On large clusters the kernel's default ARP cache limits may be exceeded,
causing node identification to fail. See
:ref:`arp-cache-overflow-on-large-clusters` for tuning guidance.
The ``{file}`` component determines what is served: The ``{file}`` component determines what is served:
* ``shim.efi``: Serves the ``shim.efi`` binary extracted from the node's * ``shim.efi``: Serves the ``shim.efi`` binary extracted from the node's

50
userdocs/troubleshooting/known-issues.rst
View File

@@ -99,3 +99,53 @@ gigabytes. Workarounds exist for these issues in most circumstances:
If you are still getting "Not enough memory" or "No space left on device" If you are still getting "Not enough memory" or "No space left on device"
errors, try disabling any "memory hole" features or updating your system BIOS errors, try disabling any "memory hole" features or updating your system BIOS
or firmware. or firmware.
.. _arp-cache-overflow-on-large-clusters:
ARP Cache Overflow on Large Clusters
=====================================
On clusters with many nodes, the Linux kernel's default ARP cache limits may be
too low, causing the Warewulf server to log:
.. code-block:: text
neighbour: arp_cache: neighbor table overflow!
The kernel manages the ARP cache with three garbage-collection thresholds:
* ``net.ipv4.neigh.default.gc_thresh1`` -- no garbage collection below this
count (default: 128)
* ``net.ipv4.neigh.default.gc_thresh2`` -- garbage collection is triggered above
this count (default: 512)
* ``net.ipv4.neigh.default.gc_thresh3`` -- hard limit; entries are dropped above
this count (default: 1024)
This is particularly relevant for Warewulf because ``warewulfd`` identifies nodes
by looking up their IP address in the kernel's ARP cache (``/proc/net/arp``) during
provisioning. If cache entries for cluster nodes are evicted, node identification
can fail.
Increase the thresholds on the Warewulf server. As a starting point:
.. code-block:: shell
sysctl -w net.ipv4.neigh.default.gc_thresh2=1024
sysctl -w net.ipv4.neigh.default.gc_thresh3=2048
For larger clusters (hundreds of nodes or more), higher values may be needed:
.. code-block:: shell
sysctl -w net.ipv4.neigh.default.gc_thresh2=2048
sysctl -w net.ipv4.neigh.default.gc_thresh3=4096
To make the change persistent across reboots, create a file in ``/etc/sysctl.d/``:
.. code-block:: shell
cat > /etc/sysctl.d/90-warewulf-arp.conf << 'EOF'
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh3 = 4096
EOF
sysctl -p /etc/sysctl.d/90-warewulf-arp.conf