Storage and resource configuration

Storage for data volumes

Druid uses S3 or HDFS for deep storage, so no extra PersistentVolumeClaims have to be specified.

Resource Requests

Stackable operators handle resource requests in a sligtly different manner than Kubernetes. Resource requests are defined on role or group level. See Roles and role groups for details on these concepts. On a role level this means that e.g. all workers will use the same resource requests and limits. This can be further specified on role group level (which takes priority to the role level) to apply different resources.

This is an example on how to specify CPU and memory resources using the Stackable Custom Resources:

---
apiVersion: example.stackable.tech/v1alpha1
kind: ExampleCluster
metadata:
  name: example
spec:
  workers: # role-level
    config:
      resources:
        cpu:
          min: 300m
          max: 600m
        memory:
          limit: 3Gi
    roleGroups: # role-group-level
      resources-from-role: # role-group 1
        replicas: 1
      resources-from-role-group: # role-group 2
        replicas: 1
        config:
          resources:
            cpu:
              min: 400m
              max: 800m
            memory:
              limit: 4Gi

In this case, the role group resources-from-role will inherit the resources specified on the role level. Resulting in a maximum of 3Gi memory and 600m CPU resources.

The role group resources-from-role-group has maximum of 4Gi memory and 800m CPU resources (which overrides the role CPU resources).

For Java products the actual used Heap memory is lower than the specified memory limit due to other processes in the Container requiring memory to run as well. Currently, 80% of the specified memory limits is passed to the JVM.

For memory only a limit can be specified, which will be set as memory request and limit in the Container. This is to always guarantee a Container the full amount memory during Kubernetes scheduling.

If no resources are configured explicitly, the Druid operator uses the following defaults:

brokers:
  roleGroups:
    default:
      config:
        resources:
          cpu:
            min: '200m'
            max: "4"
          memory:
            limit: '2Gi'

The default values are most likely not sufficient to run a proper cluster in production. Please adapt according to your requirements.

For more details regarding Kubernetes CPU limits see: Assign CPU Resources to Containers and Pods.

Historical Resources

In addition to the cpu and memory resources described above, historical Pods also accept a storage resource with the following properties:

segmentCache - used to set the maximum size allowed for the historical segment cache locations. See the Druid documentation regarding druid.segmentCache.locations. The operator creates an emptyDir and sets the max_size of the volume to be the value of the capacity property. In addition Druid is configured to keep 7% volume size free. By default, if no segmentCache is configured, the operator will create an emptyDir with a size of 1G and freePercentage of 5.

Example historical configuration with storage resources:

historicals:
  roleGroups:
    default:
      config:
        resources:
          storage:
            segmentCache:
              # The amount of free space to subtract from the capacity setting below. Defaults to 5%
              freePercentage: 7
              emptyDir:
                # The maximum size of the volume used to store the segment cache
                capacity: 2Gi