EKS on Amazon Web Services Deployment

OpenMetadata supports the Installation and Running of Application on Elastic Kubernetes Services (EKS) through Helm Charts. However, there are some additional configurations which needs to be done as prerequisites for the same.

All the code snippets in this section assume the default namespace for kubernetes. This guide presumes you have AWS EKS Cluster already available.

Prerequisites

AWS Services for Database as RDS and Search Engine as ElasticSearch

It is recommended to use Amazon RDS and Amazon OpenSearch Service for Production Deployments. We support

Amazon RDS (MySQL) engine version 8 or higher
Amazon RDS (PostgreSQL) engine version 15 or higher
Amazon OpenSearch engine version 3.x (minimum 3.0.0, recommended 3.3.0)

When using AWS Services the SearchType Configuration for elastic search should be opensearch, for both cases ElasticSearch and OpenSearch, as you can see in the ElasticSearch configuration example below.

We recommend

Amazon RDS to be in Multiple Availability Zones.
Amazon OpenSearch (or ElasticSearch) Service with Multiple Availability Zones with minimum 2 Nodes.

Make sure to increase sort_buffer_size (for MySQL) or work_mem (for PostgreSQL) to the recommended value of 20MB or more using the database parameter group setting. This is especially important when running migrations to prevent Out of Sort Memory Error. You can revert the setting once the migrations are complete.

Kubernetes Orchestrator Configuration (Recommended)

Starting with OpenMetadata 1.12, we recommend using the Kubernetes native orchestrator for running ingestion pipelines. This eliminates the need for Apache Airflow and simplifies your deployment.

The Kubernetes orchestrator runs ingestion pipelines as native K8s Jobs and CronJobs. For full documentation on features, configuration options, and troubleshooting, see the Kubernetes Orchestrator Guide.

The recommended OMJob Operator approach requires installing Custom Resource Definitions (CRDs), which needs elevated cluster permissions. If your cluster policies don’t allow CRDs, you can disable the operator by setting useOMJobOperator: false and omjobOperator.enabled: false in your values file to use native K8s Jobs instead.

OpenMetadata Values Configuration

Create your openmetadata-values.yaml with the following configuration:

# openmetadata-values.yaml
openmetadata:
  config:
    # Database configuration
    elasticsearch:
      host: <AMAZON_OPENSEARCH_SERVICE_ENDPOINT_WITHOUT_HTTPS>
      searchType: opensearch
      port: 443
      scheme: https
      connectionTimeoutSecs: 5
      socketTimeoutSecs: 60
      keepAliveTimeoutSecs: 600
      batchSize: 10
      auth:
        enabled: true
        username: <AMAZON_OPENSEARCH_USERNAME>
        password:
          secretRef: elasticsearch-secrets
          secretKey: openmetadata-elasticsearch-password
    database:
      host: <AMAZON_RDS_ENDPOINT>
      port: 3306
      driverClass: com.mysql.cj.jdbc.Driver
      dbScheme: mysql
      dbUseSSL: true
      databaseName: <RDS_DATABASE_NAME>
      auth:
        username: <RDS_DATABASE_USERNAME>
        password:
          secretRef: mysql-secrets
          secretKey: openmetadata-mysql-password

    # Kubernetes Orchestrator configuration
    pipelineServiceClientConfig:
      enabled: true
      type: "k8s"
      metadataApiEndpoint: http://openmetadata:8585/api

      k8s:
        ingestionImage: "docker.getcollate.io/openmetadata/ingestion-base:1.12.0"
        useOMJobOperator: true

# Enable the OMJob Operator (recommended for production)
omjobOperator:
  enabled: true
  image:
    repository: docker.getcollate.io/openmetadata/omjob-operator
    tag: "1.12.0"

For advanced configuration options such as resource limits, job lifecycle settings, failure diagnostics, RBAC, and security contexts, see the Kubernetes Orchestrator Guide.

Create Kubernetes Secrets

Create the required secrets for RDS and OpenSearch:

# Database secret
kubectl create secret generic mysql-secrets \
  --from-literal=openmetadata-mysql-password=<YOUR_RDS_PASSWORD>

# OpenSearch secret
kubectl create secret generic elasticsearch-secrets \
  --from-literal=openmetadata-elasticsearch-password=<YOUR_OPENSEARCH_PASSWORD>

Deploy OpenMetadata

# Add the OpenMetadata Helm repository
helm repo add open-metadata https://helm.open-metadata.org/
helm repo update

# Install OpenMetadata (no dependencies chart needed with K8s orchestrator)
helm install openmetadata open-metadata/openmetadata \
  --values openmetadata-values.yaml

With the Kubernetes orchestrator, you don’t need to deploy the openmetadata-dependencies chart that includes Airflow. This significantly simplifies your deployment.

Verify the Deployment

# Check pods are running
kubectl get pods

# Check the K8s orchestrator health in OpenMetadata UI
# Navigate to Settings → Preferences → Health

If you prefer to use Apache Airflow as the orchestrator for your EKS deployment, see the Airflow on EKS guide.

​EKS on Amazon Web Services Deployment

​Prerequisites

​AWS Services for Database as RDS and Search Engine as ElasticSearch

​Kubernetes Orchestrator Configuration (Recommended)

​OpenMetadata Values Configuration

​Create Kubernetes Secrets

​Deploy OpenMetadata

​Verify the Deployment

EKS on Amazon Web Services Deployment

Prerequisites

AWS Services for Database as RDS and Search Engine as ElasticSearch

Kubernetes Orchestrator Configuration (Recommended)

OpenMetadata Values Configuration

Create Kubernetes Secrets

Deploy OpenMetadata

Verify the Deployment