Documentation
Contact Us
Home Getting Started
Platform
Concepts Demos Tutorials CRD Reference Release Notes
Operators
Overview Apache Airflow Apache Druid Apache HBase Apache Hadoop HDFS Apache Hive Apache Kafka Apache NiFi Apache Spark on K8S Apache Superset Trino Apache ZooKeeper OpenPolicyAgent Commons Secret Listener
Management
Cockpit stackablectl
Policies and Licenses
Product Information Policies Licenses
Contribute
nightly 23.11 23.7 23.4 23.1

Stackable Documentation

    • Quickstart
    • Kubernetes
    • Getting Started
    • Concepts
      • Roles and role groups
      • Service exposition
      • Service discovery ConfigMap
      • Logging
      • Authentication
      • OPA authorization
      • Product image selection
      • PersistentVolumeClaim usage
      • Resource management
      • S3 resources
      • TLS server verification
      • Overrides
      • Duration format
      • Operations
        • Cluster operations
        • Allowed Pod disruptions
        • Pod placement
        • Graceful shutdown
    • Demos
      • airflow-scheduled-job
      • data-lakehouse-iceberg-trino-spark
      • hbase-hdfs-cycling-data
      • jupyterhub-pyspark-hdfs-anomaly-detection-taxi-data
      • logging
      • nifi-kafka-druid-earthquake-data
      • nifi-kafka-druid-water-level-data
      • signal-processing
      • spark-k8s-anomaly-detection-taxi-data
      • trino-iceberg
      • trino-taxi-data
    • Tutorials
      • Authentication with OpenLDAP
      • Logging with a Vector log aggregator
      • Enabling verification of image signatures
    • Reference
      • AuthenticationClass
      • S3 resources
    • Operators
      • Supported Product Versions
      • Monitoring
      • Apache Airflow
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Required external components
        • Usage guide
          • Mounting DAGs
          • Applying Custom Resources
          • Service exposition with ListenerClasses
          • Resource Requests
          • Security
          • Log aggregation
          • Monitoring
          • Configuration & Environment Overrides
          • Operations
            • Cluster Operation
            • Pod placement
            • Allowed Pod disruptions
            • Graceful shutdown
      • Apache Druid
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Required external components
        • Usage guide
          • Service exposition with ListenerClasses
          • Ingestion
          • Deep storage configuration
          • Storage and resource configuration
          • Security
          • Discovery
          • Log aggregation
          • Monitoring
          • Configuration & Environment Overrides
          • Operations
            • Cluster operation
            • Pod placement
            • Allowed Pod disruptions
            • Graceful shutdown
      • Apache HBase
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Usage guide
          • Service exposition with ListenerClasses
          • Resource requests
          • Using Apache Phoenix
          • Compression support
          • Discovery
          • Log aggregation
          • Monitoring
          • Configuration overrides
          • Operations
            • Cluster operation
            • Pod placement
            • Allowed Pod disruptions
            • Graceful shutdown
      • Apache Hadoop HDFS
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Concepts
          • Discovery
        • Usage guide
          • Service exposition with ListenerClasses
          • Security
          • Resources
          • Scaling
          • FUSE
          • Logging & log aggregation
          • Monitoring
          • Configuration & Environment Overrides
          • Operations
            • Cluster Operation
            • Pod placement
            • HDFS Rack Awareness
            • Allowed Pod disruptions
            • Graceful shutdown
      • Apache Hive
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Concepts
          • Discovery
        • Required external components
        • Usage guide
          • Service exposition with ListenerClasses
          • Data storage backends
          • Derby example
          • Log aggregation
          • Monitoring
          • Configuration & environment overrides
          • Operations
            • Cluster operation
            • Pod placement
            • Allowed Pod disruptions
            • Graceful shutdown
      • Apache Kafka
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Usage guide
          • Storage and resource configuration
          • Security
          • Discovery
          • Monitoring
          • Log aggregation
          • Configuration & Environment Overrides
          • Operations
            • Cluster operation
            • Pod placement
            • Allowed Pod disruptions
            • Graceful shutdown
            • Cluster ID
      • Apache NiFi
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Usage guide
          • Service exposition with ListenerClasses
          • Connecting NiFi to ZooKeeper
          • Adding External Files to the NiFi Servers
          • Loading Custom Components
          • Expose NiFi processor ports
          • Security
          • Resource configuration
          • Log aggregation
          • Monitoring
          • Updating NiFi
          • Configuration & Environment Overrides
          • Operations
            • Cluster operation
            • Pod placement
            • Allowed Pod disruptions
            • Graceful shutdown
      • Apache Spark on K8S
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Usage guide
          • Service exposition with ListenerClasses
          • Job Dependencies
          • Resource Requests
          • S3 bucket specification
          • Spark History Server
          • Examples
          • Operations
            • Pod Placement
            • Allowed Pod disruptions
            • Graceful shutdown
        • CRD reference
      • Apache Superset
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Required external components
        • Usage guide
          • Service exposition with ListenerClasses
          • Storage and resource configuration
          • Security
          • Connecting Apache Druid clusters
          • Monitoring
          • Log aggregation
          • Configuration & Environment Overrides
          • Operations
            • Cluster Operation
            • Pod Placement
            • Allowed Pod disruptions
            • Graceful shutdown
      • Trino
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Concepts
        • Usage guide
          • Connect to Trino
          • Service exposition with ListenerClasses
          • Configuration
          • S3 connection specification
          • Security
          • Monitoring
          • Log aggregation
          • Testing Trino with Hive and S3
          • Catalogs
            • Black Hole
            • Generic
            • Google sheets
            • Apache Hive
            • Apache Iceberg
            • TPC-DS
            • TPC-H
          • Operations
            • Cluster Operation
            • Pod placement
            • Allowed Pod disruptions
            • Graceful shutdown
      • Apache ZooKeeper
        • Getting started
          • Installation
          • First steps
        • Configuration
        • Concepts
          • ZNodes
          • Discovery Profiles
        • Usage guide
          • Service exposition with ListenerClasses
          • Encryption
          • Authentication
          • Storage and resource configuration
          • Monitoring
          • Log aggregation
          • Using multiple role groups
          • Isolating clients with ZNodes
          • Configuration and environment overrides
          • Operations
            • Cluster Operation
            • Pod placement
            • Allowed Pod disruptions
            • Graceful shutdown
      • OpenPolicyAgent
        • Getting started
          • Installation
          • First steps
        • Concepts
          • Discovery
        • Configuration
        • Implementation Notes
        • Usage guide
          • Service exposition with ListenerClasses
          • Defining policies
          • Resource requests
          • Log aggregation
          • Monitoring
          • Configuration & Environment Overrides
          • Operations
            • Cluster Operation
            • Allowed Pod disruptions
            • Graceful shutdown
      • Commons
        • Installation
        • Configuration
        • Usage
        • Concepts
          • Restarter
          • Pod Enricher
      • Listener
        • Building the Operator
        • Installation
        • Usage
        • Concepts
          • Listener
          • ListenerClass
          • Volume
        • Security
      • Secret
        • Building the Operator
        • Installation
        • Configuration
        • Usage
        • Concepts
          • SecretClass
          • Scope
          • Volume
        • Security
        • Troubleshooting
    • Contributor’s Guide
      • Steps to contribute
      • Testing your Code on Kubernetes
      • Development dashboard
      • Documentation style guide
      • Implementation guidelines
        • Logging aggregation implementation guidelines
        • Service discovery implementation guidelines
        • OPA connection implementation guidelines
      • Architectural Decision Records
        • Current
          • ADR001: Use English as Documentation Language
          • ADR002: Use Multiple Repositories instead of one Large Repository
          • ADR003: Use RTC as Review Mechanism for Changes
          • ADR004: Use Rust as programming language for the agent
          • ADR005: Decide on handling and location of systemd unit files
          • ADR007: Decide if Kubernetes Components Are to be Reused for Stackable
          • ADR008: Allow Reuse of Existing Kubernetes Operators
          • ADR009: Assigning Services to Nodes
          • ADR010: Expressing one-shot commands in a Kubernetes-native way
          • ADR011: Directory Structure Used by Stackable Components on Managed Hosts
          • ADR012: Authentication token management
          • ADR013: Supported Kubernetes versions
          • ADR014: User Authentication for Products
          • ADR015: How Should Operators Use Values from ConfigMaps & Secrets
          • ADR016: Representation of S3 Buckets in CRDs
          • ADR017: TLS authentication
          • ADR018: Product Image Versioning
          • ADR019: Trino catalog definitions
          • ADR020: Trino catalog usage
          • ADR021: Initial Version of Stackable Stacks Functionality
          • ADR022: Spark history server
          • ADR023: Product image selection
          • ADR024: How to provide stable out-of-cluster access to products
          • ADR025: Logging and Log Aggregation Architecture
          • ADR026: Affinities
          • ADR027: Resource Status
          • ADR028: Automatic stackable version selection
          • ADR029: Standardize database connections
          • ADR030: Allowed Pod disruptions
          • ADRXXX: Resource Labels and Namespacing
        • Deprecated
          • Use xxx as storage backend for the orchestrator
        • Drafts
          • Choose Authorization Engine
    • Release notes
    • Product information
    • Lifecycle policies
    • Licenses
Stackable Documentation 23.11
  • SDP Management
    • nightly
  • Stackable Documentation
    • nightly
    • 23.11
    • 23.7
    • 23.4
    • 23.1
  • Stackable Documentation
  • Operators
  • Apache Spark on K8S
  • Usage guide
Edit this Page

Usage guide

Learn how to load your own Job Dependencies or configure an S3 connection to access data. Have a look at the Examples to learn more about different usage scenarios.

your data, your platform

Company

  • About Us
  • Blog
  • Jobs

Support

  • Contact Us
  • Imprint
  • Data Protection

Get in touch

  • info@stackable.de
  • +49 4103 926 3100
© 2022 Stackable.

Apache, Apache Kafka®, Kafka, and the Kafka logo, Apache Druid, Druid, and the Druid logo, Apache ZooKeeper™, ZooKeeper, and the Druid logo, Apache Hive™, Hive, and the Hive logo, Apache Spark™, Spark, and the Spark logo, Apache Airflow, Airflow, and the Airflow logo, Apache HBase®, HBase, and the HBase logo, Apache NiFi, NiFi, and the NiFi logo, Apache Superset, Superset, the Superset logo, Apache Hadoop® HDFS, Apache Hadoop, Hadoop and the Hadoop logo, Apache Phoenix™, Phoenix and the Phoenix-Logo, Apache Iceberg, Iceberg and the Iceberg-Logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. Open Policy Agent (OPA) is a Cloud Native Computing Foundation graduated project. Licensed under the Apache License, Version 2.0. Trino is open source software licensed under the Apache License 2.0 and supported by the Trino Software Foundation. MinIO is a [“registered”, if applicable] trademark of the MinIO Corporation. All other products or name brands are trademarks of their respective holders. All product and service names used in this website are for identification purposes only and do not imply endorsement.