kelvin/warewulf
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Initial cut for removing the old API.

Browse Source 
Remove API reverse proxy server and google dependencies.
Remove API config.
Remove API client.
Remove API GRPC server.
Remove unused ImageCopy function. image.Duplicate is used.
Update Makefile to avoid protobuf builds.

Signed-off-by: Matt Hink <mhink@ciq.com>
This commit is contained in:
Matt Hink
2025-04-21 16:00:48 -07:00
committed by Jonathon Anderson
parent edc0e151c3
commit be73ac056a
24 changed files with 4 additions and 2271 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
internal/app/api/API.md
View File

@@ -1,32 +0,0 @@
We need an API for Warewulf in order to automate around it. The initial version of the API will look a lot like wwctl. wwctl will call the API by direct function call so that we do not need to maintain separate code paths, nor is the API required.
wwctl calls the API via direct function call, but we can change that to a gprc or REST call to a Warewulf server in the future.
The API currently contains:
wwapid WareWulf API Daemon. A grpc server with mTLS auth.
wwapic WareWulf API Client. A grpc client with mTLS auth. It's just a sample that reads the version from the server.
wwapird WareWulf API Rest Daemon. A http REST reverse proxy to wwapid.
There are also sample insecure and secure curls to test with.
Implemented Functionality:
wwctl node add
wwctl node delete
wwctl node list
wwctl node set
wwctl node status
wwctl image build
wwctl image delete
wwctl image import
wwctl image list
wwctl image show
wwctl image copy
Some notes on the files:
Logic that was in wwctl has moved to warewulf/internal/pkg/api. wwctl just calls the API. wwctl functionality is unchanged.
Everything under the /google directory is copied google proto code to stand up wwapird. It's a bit strange to copy in the code, but that is currently how it's done.
Everything in warewulf/internal/pkg/api/routes/wwapiv1 is generated code.
There are some loose ends here, such as no service installers. I just ran off the command line for development.
The Makefile could use improvement. I'm building by make clean setup proto all build wwapid wwapic wwapird ; echo $?
I copied the configs from warewulf/etc to /usr/local/etc/warewulf manually.

4
internal/app/api/wwapic/README.md
View File

@@ -1,4 +0,0 @@
# wwapiclient is intended as and example grpc wwapid client.
Run wwapid to start the server
```./wwapic``` will get the version.
This works with or without mTLS.

90
internal/app/api/wwapic/wwapic.go
View File

@@ -1,90 +0,0 @@
package main
import (
"context"
"crypto/tls"
"crypto/x509"
"fmt"
"log"
"os"
"path"
"time"
"github.com/warewulf/warewulf/internal/pkg/api/apiconfig"
"github.com/warewulf/warewulf/internal/pkg/api/routes/wwapiv1"
warewulfconf "github.com/warewulf/warewulf/internal/pkg/config"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/protobuf/types/known/emptypb"
)
// wwapic is intended as a sample wwapi client.
func main() {
log.Println("Client running")
conf := warewulfconf.Get()
// Read the config file.
config, err := apiconfig.NewClient(path.Join(conf.Paths.Sysconfdir, "warewulf/wwapic.conf"))
if err != nil {
log.Fatalf("err: %v", err)
}
var opts []grpc.DialOption
if config.TlsConfig.Enabled {
// Load the client cert and its key
clientCert, err := tls.LoadX509KeyPair(config.TlsConfig.Cert, config.TlsConfig.Key)
if err != nil {
log.Fatalf("Failed to load client cert and key. %s.", err)
}
// Load the CA cert.
var cacert []byte
cacert, err = os.ReadFile(config.TlsConfig.CaCert)
if err != nil {
log.Fatalf("Failed to load cacert. err: %s\n", err)
}
// Put the CA cert into the cert pool.
certPool := x509.NewCertPool()
if !certPool.AppendCertsFromPEM(cacert) {
log.Fatalf("Failed to append CA cert to certificate pool. %s.", err)
}
// Create the TLS configuration
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{clientCert},
RootCAs: certPool,
MinVersion: tls.VersionTLS13,
MaxVersion: tls.VersionTLS13,
}
// Create TLS credentials from the TLS configuration
creds := credentials.NewTLS(tlsConfig)
opts = append(opts, grpc.DialOption(grpc.WithTransportCredentials(creds)))
} else {
opts = append(opts, grpc.WithTransportCredentials(insecure.NewCredentials()))
}
conn, err := grpc.NewClient(fmt.Sprintf("%s:%v", config.ApiConfig.Server, config.ApiConfig.Port), opts...)
if err != nil {
log.Fatalln(err)
}
defer conn.Close()
client := wwapiv1.NewWWApiClient(conn)
request := &emptypb.Empty{}
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
response, err := client.Version(ctx, request)
if err != nil {
log.Fatalln(err)
}
log.Printf("Version Response: %v\n", response)
}

7
internal/app/api/wwapid/README.md
View File

@@ -1,7 +0,0 @@
# Warewulf API Daemon
wwapid is a grpc service serving the Warewulf API. For v1, the intent is to serve the same interface as wwctl serves. For this preview PR, we are serving much of ```wwctl node``` and ```wwctl image```.
Initial security is by mTLS. For development we are generating our own keys. A good tutorial for this is here: https://www.handracs.info/blog/grpcmtlsgo/
The configuration file is wwapid.conf.

356
internal/app/api/wwapid/wwapid.go
View File

@@ -1,356 +0,0 @@
package main
import (
"bufio"
"context"
"crypto/tls"
"crypto/x509"
"fmt"
"log"
"net"
"os"
"path"
"github.com/warewulf/warewulf/internal/pkg/api/apiconfig"
"github.com/warewulf/warewulf/internal/pkg/api/image"
apinode "github.com/warewulf/warewulf/internal/pkg/api/node"
"github.com/warewulf/warewulf/internal/pkg/api/routes/wwapiv1"
warewulfconf "github.com/warewulf/warewulf/internal/pkg/config"
"github.com/warewulf/warewulf/internal/pkg/version"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
)
type apiServer struct {
wwapiv1.UnimplementedWWApiServer
}
var apiPrefix string
var apiVersion string
func main() {
log.Println("Server running")
conf := warewulfconf.Get()
// Read the config file.
config, err := apiconfig.NewServer(path.Join(conf.Paths.Sysconfdir, "warewulf/wwapid.conf"))
if err != nil {
log.Fatalf("err: %v", err)
}
// Pull out config variables.
apiPrefix = config.ApiConfig.Prefix
apiVersion = config.ApiConfig.Version
servicePort := config.ApiConfig.Port
portString := fmt.Sprintf(":%d", servicePort)
var opts []grpc.ServerOption
if !config.TlsConfig.Enabled {
insecureMode()
} else {
// Setup TLS.
serverCert, err := tls.LoadX509KeyPair(config.TlsConfig.Cert, config.TlsConfig.Key)
if err != nil {
log.Fatalf("Failed to load server cert and key. err: %s\n", err)
}
// Load the CA cert.
var cacert []byte
cacert, err = os.ReadFile(config.TlsConfig.CaCert)
if err != nil {
log.Fatalf("Failed to load cacert. err: %s\n", err)
}
// Put the CA cert into the cert pool.
certPool := x509.NewCertPool()
if !certPool.AppendCertsFromPEM(cacert) {
log.Fatalf("Failed to append CA cert to certificate pool. %s.", err)
}
// Create the TLS configuration
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{serverCert},
RootCAs: certPool,
ClientCAs: certPool,
MinVersion: tls.VersionTLS13,
MaxVersion: tls.VersionTLS13,
}
// Create TLS credentials from the TLS configuration
creds := credentials.NewTLS(tlsConfig)
opts = []grpc.ServerOption{grpc.Creds(creds)}
}
listen, err := net.Listen("tcp", portString)
if err != nil {
log.Fatalln(err)
}
defer func() {
listen.Close()
}()
grpcServer := grpc.NewServer(opts...)
wwapiv1.RegisterWWApiServer(grpcServer, &apiServer{})
log.Fatalln(grpcServer.Serve(listen))
}
// private helpers
// insecureMode creates a blocking prompt for customers running wwapid in insecure mode.
// It's a deterrent. Setup TLS.
func insecureMode() {
fmt.Println("*** Running wwapid in INSECURE mode. *** THIS IS DANGEROUS! *** Enter y to continue. ***")
reader := bufio.NewReader(os.Stdin)
result, err := reader.ReadString('\n')
if err != nil {
fmt.Printf("Fatal error: %v\n", err)
}
if !(result == "y\n") {
os.Exit(1)
}
fmt.Printf("wwapid running IN INSECURE MODE\n")
}
// Api implementation.
// ImageBuild builds one or more images.
func (s *apiServer) ImageBuild(ctx context.Context, request *wwapiv1.ImageBuildParameter) (response *wwapiv1.ImageListResponse, err error) {
// Parameter checks.
if request == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request")
}
if request.ImageNames == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request.ImageNames")
}
// Build the image.
err = image.ImageBuild(request)
if err != nil {
return
}
// Return the built images. (A REST POST returns what is modified.)
var images []*wwapiv1.ImageInfo
images, err = image.ImageList()
if err != nil {
return
}
response = &wwapiv1.ImageListResponse{}
for i := 0; i < len(images); i++ {
for j := 0; j < len(request.ImageNames); j++ {
if images[i].Name == request.ImageNames[j] {
response.Images = append(response.Images, images[i])
}
}
}
return
}
// ImageCopy duplicates an image.
func (s *apiServer) ImageCopy(ctx context.Context, request *wwapiv1.ImageCopyParameter) (response *emptypb.Empty, err error) {
// Parameter checks.
if request == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request")
}
err = image.ImageCopy(request)
return
}
// ImageDelete deletes one or more images from Warewulf.
func (s *apiServer) ImageDelete(ctx context.Context, request *wwapiv1.ImageDeleteParameter) (response *emptypb.Empty, err error) {
// Parameter checks.
if request == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request")
}
if request.ImageNames == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request.ImageNames")
}
err = image.ImageDelete(request)
return
}
func (s *apiServer) ImageImport(ctx context.Context, request *wwapiv1.ImageImportParameter) (response *wwapiv1.ImageListResponse, err error) {
// Import the image.
var imageName string
imageName, err = image.ImageImport(request)
if err != nil {
return
}
// Return the imported image to the client.
var images []*wwapiv1.ImageInfo
images, err = image.ImageList()
if err != nil {
return
}
// Image name may have been shimmed in during import,
// which is why ImageImport returns it.
for i := 0; i < len(images); i++ {
if imageName == images[i].Name {
response = &wwapiv1.ImageListResponse{
Images: []*wwapiv1.ImageInfo{images[i]},
}
return
}
}
return
}
// ImageList returns details about images.
func (s *apiServer) ImageList(ctx context.Context, request *emptypb.Empty) (response *wwapiv1.ImageListResponse, err error) {
var images []*wwapiv1.ImageInfo
images, err = image.ImageList()
if err != nil {
return
}
response = &wwapiv1.ImageListResponse{
Images: images,
}
return
}
// ImageShow returns details about images.
func (s *apiServer) ImageShow(ctx context.Context, request *wwapiv1.ImageShowParameter) (response *wwapiv1.ImageShowResponse, err error) {
// Parameter checks.
if request == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request")
}
return image.ImageShow(request)
}
// NodeAdd adds one or more nodes for management by Warewulf and returns the added nodes.
func (s *apiServer) NodeAdd(ctx context.Context, request *wwapiv1.NodeAddParameter) (response *wwapiv1.NodeListResponse, err error) {
// Parameter checks.
if request == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request")
}
if request.NodeNames == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request.NodeNames")
}
// Add the node(s).
err = apinode.NodeAdd(request)
if err != nil {
return
}
// Return the added nodes as per REST.
return s.nodeListInternal(request.NodeNames)
}
// NodeDelete deletes one or more nodes for removal of management by Warewulf.
func (s *apiServer) NodeDelete(ctx context.Context, request *wwapiv1.NodeDeleteParameter) (response *emptypb.Empty, err error) {
response = new(emptypb.Empty)
// Parameter checks.
if request == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request")
}
if request.NodeNames == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request.NodeNames")
}
err = apinode.NodeDelete(request)
return
}
// NodeList returns details about zero or more nodes.
func (s *apiServer) NodeList(ctx context.Context, request *wwapiv1.NodeNames) (response *wwapiv1.NodeListResponse, err error) {
// Parameter checks. request.NodeNames can be nil.
if request == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request")
}
// Perform the list.
return s.nodeListInternal(request.NodeNames)
}
// NodeSet updates fields for zero or more nodes and returns the updated nodes.
func (s *apiServer) NodeSet(ctx context.Context, request *wwapiv1.ConfSetParameter) (response *wwapiv1.NodeListResponse, err error) {
// Parameter checks.
if request == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request")
}
if request.ConfList == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request.NodeNames")
}
// Perform the NodeSet.
err = apinode.NodeSet(request)
if err != nil {
return
}
// Return the updated nodes as per REST.
return s.nodeListInternal(request.ConfList)
}
func (s *apiServer) NodeStatus(ctx context.Context, request *wwapiv1.NodeNames) (response *wwapiv1.NodeStatusResponse, err error) {
// Parameter checks. request.NodeNames can be nil.
if request == nil {
return response, status.Errorf(codes.InvalidArgument, "nil request")
}
return apinode.NodeStatus(request.NodeNames)
}
// Version returns the versions of the wwapiv1 and warewulf as well as the api
// prefix for http routes.
func (s *apiServer) Version(ctx context.Context, request *emptypb.Empty) (response *wwapiv1.VersionResponse, err error) {
response = &wwapiv1.VersionResponse{
ApiPrefix: apiPrefix,
ApiVersion: apiVersion,
WarewulfVersion: version.GetVersion(),
}
return
}
// Private helpers.
// nodeListInternal calls NodeList and returns NodeListResponse.
// This does not contain parameter checks.
func (s *apiServer) nodeListInternal(nodeNames []string) (response *wwapiv1.NodeListResponse, err error) {
var nodes []*wwapiv1.NodeInfo
/*
nodes, err = apinode.NodeList(nodeNames)
if err != nil {
return
}
*/
response = &wwapiv1.NodeListResponse{
Nodes: nodes,
}
return
}

9
internal/app/api/wwapird/README.md
View File

@@ -1,9 +0,0 @@
# Warewulf API Rest Daemon
wwapird is a client of wwapid serving a REST interface for the Warewulf API. For v1, the intent is to serve the same interface as wwctl serves. For this preview PR, we are serving much of ```wwctl node``` and ```wwctl image```.
Initial security is by mTLS. For development we are generating our own keys. A good tutorial for this is here: https://www.handracs.info/blog/grpcmtlsgo/
The configuration file is wwapird.conf.
Sample cURLs with and without mTLS are in curl.sh. This file is just a scratchpad for examples.

113
internal/app/api/wwapird/wwapird.go
View File

@@ -1,113 +0,0 @@
package main
import (
"context"
"crypto/tls"
"crypto/x509"
"os"
"flag"
"fmt"
"log"
"net/http"
"github.com/golang/glog"
"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/credentials/insecure"
"github.com/warewulf/warewulf/internal/pkg/api/apiconfig"
warewulfconf "github.com/warewulf/warewulf/internal/pkg/config"
gw "github.com/warewulf/warewulf/internal/pkg/api/routes/wwapiv1"
"path"
)
func run() error {
log.Println("test0")
conf := warewulfconf.Get()
// Read the config file.
config, err := apiconfig.NewClientServer(path.Join(conf.Paths.Sysconfdir, "warewulf/wwapird.conf"))
if err != nil {
glog.Fatalf("Failed to read config file, err: %v", err)
}
grpcServerEndpoint := fmt.Sprintf("%s:%v", config.ClientApiConfig.Server, config.ClientApiConfig.Port)
httpServerEndpoint := fmt.Sprintf(":%v", config.ServerApiConfig.Port)
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
defer cancel()
// Register gRPC server endpoint (we are the client)
// Note: Make sure the gRPC server is running properly and accessible
mux := runtime.NewServeMux()
var opts []grpc.DialOption
if config.ClientTlsConfig.Enabled {
// Load the client cert and its key
clientCert, err := tls.LoadX509KeyPair(config.ClientTlsConfig.Cert, config.ClientTlsConfig.Key)
if err != nil {
log.Fatalf("Failed to load client cert and key. %s.", err)
}
// Load the CA cert.
var cacert []byte
cacert, err = os.ReadFile(config.ClientTlsConfig.CaCert)
if err != nil {
log.Fatalf("Failed to load cacert. err: %s\n", err)
}
// Put the CA cert into the cert pool.
certPool := x509.NewCertPool()
if !certPool.AppendCertsFromPEM(cacert) {
log.Fatalf("Failed to append CA cert to certificate pool. %s.", err)
}
// Create the TLS configuration
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{clientCert},
RootCAs: certPool,
MinVersion: tls.VersionTLS13,
MaxVersion: tls.VersionTLS13,
}
// Create TLS credentials from the TLS configuration
creds := credentials.NewTLS(tlsConfig)
opts = append(opts, grpc.DialOption(grpc.WithTransportCredentials(creds)))
} else {
opts = []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}
}
err = gw.RegisterWWApiHandlerFromEndpoint(ctx, mux, grpcServerEndpoint, opts)
if err != nil {
return err
}
// Start HTTP server (and proxy calls to gRPC server endpoint)
if config.ServerTlsConfig.Enabled {
return http.ListenAndServeTLS(
httpServerEndpoint,
config.ServerTlsConfig.ConcatCert,
config.ServerTlsConfig.Key,
mux)
}
// Insecure
return http.ListenAndServe(httpServerEndpoint, mux)
}
func main() {
flag.Parse() // Pretty sure glog wants this.
defer glog.Flush()
if err := run(); err != nil {
glog.Fatal(err)
}
}

57
internal/app/api/wwapird_curls/curls.sh
View File

@@ -1,57 +0,0 @@
#! /usr/bin/env bash
# This file is a scratchpad for curling wwapird.
# version
curl http://localhost:9871/version
# secure version
curl --cacert /usr/local/etc/warewulf/keys/cacert.pem \
--key /usr/local/etc/warewulf/keys/client.key \
--cert /usr/local/etc/warewulf/keys/client.pem \
https://localhost:9871/version
# image list all
curl http://localhost:9871/v1/image
# image import
curl -d '{"source": "docker://ghcr.io/warewulf/warewulf-rockylinux:8", "name": "rocky-8", "update": true, "default": true}' -H "Content-Type: application/json" -X POST http://localhost:9871/v1/image
# image delete
curl -X DELETE http://localhost:9871/v1/image?imageNames=rocky-8
# image build
curl -d '{"imageNames": ["rocky-8"], "force": true}' -H "Content-Type: application/json" -X POST http://localhost:9871/v1/imagebuild
# node list all
curl http://localhost:9871/v1/node
# node list one
curl http://localhost:9871/v1/node?nodeNames=testnode1 # this works! case sensitive
# This is a list of testnode[1-2] with URL escapes.
curl http://localhost:9871/v1/node?nodeNames=testnode%5B1-2%5D
# node add single discoverable node
curl -d '{"nodeNames": ["testApiNode0"], "discoverable": true}' -H "Content-Type: application/json" -X POST http://localhost:9871/v1/node
curl -d '{"nodeNames": ["testApiNode1"], "discoverable": true}' -H "Content-Type: application/json" -X POST http://localhost:9871/v1/node
# list the node we just added
curl http://localhost:9871/v1/node?nodeNames=testApiNode0
# This gets me a little farther, but still no param data:
curl -d '{"nodeNames": ["testApiNode0"], "ipmiIpAddr": "10.0.8.220", "updateMask": "ipmiIpAddr,nodeNames"}' -H "Content-Type: application/json" -X PATCH http://localhost:9871/v1/node
# Node set with post:
curl -d '{"nodeNames": ["testApiNode0"], "ipmiIpaddr": "6.7.8.9"}' -H "Content-Type: application/json" -X POST http://localhost:9871/v1/nodeset
# node status
curl http://localhost:9871/v1/nodestatus
curl http://localhost:9871/v1/nodestatus?nodeNames=testApiNode0
# node delete single node
curl -X DELETE http://localhost:9871/v1/node?nodeNames=testApiNode0
curl -X DELETE http://localhost:9871/v1/node?nodeNames=testApiNode1