Derby example

Please note that the version you need to specify is not only the version of Apache Hive which you want to roll out, but has to be amended with a Stackable version as shown. This Stackable version is the version of the underlying container image which is used to execute the processes. For a list of available versions please check our image registry. It should generally be safe to simply use the latest image version that is available.

Create a single node Apache Hive Metastore cluster using Derby:

---
apiVersion: hive.stackable.tech/v1alpha1
kind: HiveCluster
metadata:
  name: simple-hive-derby
spec:
  image:
    productVersion: 3.1.3
  clusterConfig:
    database:
      connString: jdbc:derby:;databaseName=/tmp/metastore_db;create=true
      user: APP
      password: mine
      dbType: derby
  metastore:
    roleGroups:
      default:
        replicas: 1

You should not use the Derby database in production. Derby stores data locally which does not work in high availability setups (multiple replicas) and all data is lost after Pod restarts.

To create a single node Apache Hive Metastore (v3.1.3) cluster with derby and S3 access, deploy a minio (or use any available S3 bucket):

helm install minio \
    minio \
    --repo https://charts.bitnami.com/bitnami \
    --set auth.rootUser=minio-access-key \
    --set auth.rootPassword=minio-secret-key

In order to upload data to minio we need a port-forward to access the web ui.

kubectl port-forward service/minio 9001

Then, connect to localhost:9001 and login with the user minio-access-key and password minio-secret-key. Create a bucket and upload data.

Deploy the hive cluster:

---
apiVersion: hive.stackable.tech/v1alpha1
kind: HiveCluster
metadata:
  name: simple-hive-derby
spec:
  image:
    productVersion: 3.1.3
  clusterConfig:
    database:
      connString: jdbc:derby:;databaseName=/stackable/metastore_db;create=true
      user: APP
      password: mine
      dbType: derby
    s3:
      inline:
        host: minio
        port: 9000
        accessStyle: Path
        credentials:
          secretClass: simple-hive-s3-secret-class
  metastore:
    roleGroups:
      default:
        replicas: 1
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
  name: simple-hive-s3-secret-class
spec:
  backend:
    k8sSearch:
      searchNamespace:
        pod: {}
---
apiVersion: v1
kind: Secret
metadata:
  name: simple-hive-s3-secret
  labels:
    secrets.stackable.tech/class: simple-hive-s3-secret-class
stringData:
  accessKey: minio-access-key
  secretKey: minio-secret-key

To create a single node Apache Hive Metastore using PostgreSQL, deploy a PostgreSQL instance via helm.

PostgreSQL introduced a new way to encrypt its passwords in version 10. This is called scram-sha-256 and has been the default as of PostgreSQL 14. Unfortunately, Hive up until the latest 3.3.x version ships with JDBC drivers that do not support this method. You might see an error message like this: The authentication type 10 is not supported. If this is the case please either use an older PostgreSQL version or change its password_encryption setting to md5.

This installs PostgreSQL in version 10 to work around the issue mentioned above:

helm install hive bitnami/postgresql --version=12.1.5 \
--set postgresqlUsername=hive \
--set postgresqlPassword=hive \
--set postgresqlDatabase=hive

Create Hive Metastore using a PostgreSQL database

apiVersion: hive.stackable.tech/v1alpha1
kind: HiveCluster
metadata:
  name: simple-hive-postgres
spec:
  image:
    productVersion: 3.1.3
  clusterConfig:
    database:
      connString: jdbc:postgresql://hive-postgresql.default.svc.cluster.local:5432/hive
      user: hive
      password: hive
      dbType: postgres
  metastore:
    roleGroups:
      default:
        replicas: 1