Kubernetes CRD schema addition of enum from values.yaml

ONLINECAST

3 min read

·

05-10-2024

9

0

Share

Dynamically Defining Enum Values in Kubernetes Custom Resource Definitions

Problem: You want to add an enum (enumeration) type to your Kubernetes Custom Resource Definition (CRD) schema, and you want the possible values to be defined dynamically from a values.yaml file. This allows you to manage the allowed values in a separate configuration file, making your CRD schema more flexible and maintainable.

Scenario: Imagine you're developing a custom resource for managing application configurations. You want to enforce specific logging levels, such as "INFO", "DEBUG", "WARN", and "ERROR". You want to store these logging levels in your values.yaml file for easy modification.

Original Code (Using Static Enum Values):

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: myapplications.mygroup.example.com
spec:
  versions:
  - name: v1
    served: true
    storage: true
  scope: Namespaced
  names:
    plural: myapplications
    singular: myapplication
  validation:
    openAPIV3Schema:
      properties:
        loggingLevel:
          type: string
          enum:
          - "INFO"
          - "DEBUG"
          - "WARN"
          - "ERROR"

Solution and Analysis:

The problem with the above code is that the enum values are hardcoded. To dynamically define them from values.yaml, we can utilize a custom controller that reads the values.yaml file and dynamically updates the CRD schema. Here's a step-by-step approach:

1. Define Values in values.yaml:

loggingLevels:
- INFO
- DEBUG
- WARN
- ERROR

2. Create a Custom Controller:

   • The controller needs to read the values.yaml file on startup.
   • Extract the logging levels from the file.
   • Use the Kubernetes API to patch the CRD schema, updating the enum values.

   Code Example (Using Go):

   package main
   
   import (
       "context"
       "fmt"
       "io/ioutil"
       "os"
   
       metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
       "k8s.io/apimachinery/pkg/runtime/schema"
       "k8s.io/apimachinery/pkg/util/yaml"
       "k8s.io/client-go/kubernetes"
       "k8s.io/client-go/rest"
       apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
   )
   
   func main() {
       // Load values.yaml
       values, err := loadValues("values.yaml")
       if err != nil {
           panic(err)
       }
   
       // Create Kubernetes client
       config, err := rest.InClusterConfig()
       if err != nil {
           panic(err)
       }
       clientset, err := kubernetes.NewForConfig(config)
       if err != nil {
           panic(err)
       }
   
       // Get CRD
       crd, err := clientset.ApiextensionsV1().CustomResourceDefinitions().Get(context.Background(), "myapplication.mygroup.example.com", metav1.GetOptions{})
       if err != nil {
           panic(err)
       }
   
       // Update schema with dynamic values
       crd.Spec.Validation.OpenAPIV3Schema.Properties["loggingLevel"].Enum = values["loggingLevels"]
       _, err = clientset.ApiextensionsV1().CustomResourceDefinitions().Update(context.Background(), crd, metav1.UpdateOptions{})
       if err != nil {
           panic(err)
       }
   
       fmt.Println("CRD updated successfully.")
   }
   
   func loadValues(filename string) (map[string]interface{}, error) {
       data, err := ioutil.ReadFile(filename)
       if err != nil {
           return nil, err
       }
   
       var values map[string]interface{}
       err = yaml.Unmarshal(data, &values)
       if err != nil {
           return nil, err
       }
       return values, nil
   }
   

3. Deploy the Controller:

   • Create a deployment for your controller using YAML or other deployment tools.
   • Make sure the deployment mounts the values.yaml file.

Benefits:

• Flexibility: Allows you to easily change the allowed values for your enums without modifying the CRD definition.
• Maintainability: Centrally manages the allowed values in a single file.
• Configuration Management: Easily manage and update the enum values through standard configuration management tools.

Considerations:

• Complexity: Adding a custom controller increases the complexity of your deployment.
• Concurrency: Ensure your controller handles concurrency issues when updating the CRD.
• Security: Carefully consider the security implications of dynamically modifying CRD schemas.

Further Reading:

• Kubernetes API Reference
• CustomResourceDefinition Specification
• Go Client for Kubernetes
• Kubernetes CRD Best Practices

Conclusion:

Dynamically defining enum values in Kubernetes CRDs using a custom controller provides a flexible and maintainable approach for managing your custom resources. This approach allows for easy adaptation of the enum values without directly modifying the CRD definition.