Local Docker Deployment

This installation doc will help you start a OpenMetadata standalone instance on your local machine.

If you'd rather see the steps in a guided tutorial, we've got you covered! Otherwise, feel free to read the content below 👇

Please ensure your host system meets the requirements listed below. Then continue to the Procedure for installing OpenMetadata.

To check the version of Python you have, use the following command:

Docker is an open-source platform for developing, shipping, and running applications. It enables you to separate your applications from your infrastructure, so you can deliver software quickly using OS-level virtualization. It helps deliver software in packages called Containers.

To check the version of Docker you have, use the following command.

If you need to install Docker, please visit Get Docker.

You must allocate at least 6GB of memory to Docker in order to run OpenMetadata. To change the memory allocation for Docker, please visit Preferences -> Resources -> Advanced in your Docker Desktop.

The Docker compose package enables you to define and run multi-container Docker applications. The compose command integrates compose functions into the Docker platform, making them available from the Docker command-line interface ( CLI). The Python packages you will install in the procedure below use compose to deploy OpenMetadata.

  • MacOS X: Docker on MacOS X ships with compose already available in the Docker CLI.
  • Linux: To install compose on Linux systems, please visit the Docker CLI command documentation and follow the instructions.

To verify that the docker compose command is installed and accessible on your system, run the following command.

Upon running this command you should see output similar to the following.

Follow the instructions here to install docker compose version 2.0.0

  1. Run the following command to download the current stable release of Docker Compose

    This command installs Compose V2 for the active user under $HOME directory. To install Docker Compose for all users on your system, replace ~/.docker/cli-plugins with /usr/local/lib/docker/cli-plugins.

  2. Apply executable permissions to the binary

  3. Test your installation

  • Install WSL2
  • Install Ubuntu 20.04
  • Install Docker for Windows
    • Once installed, please follow the steps here and complete all the pre-requisites for a seamless installation and deployment.
    • After completion of the pre-requisites, please install python3-pip and python3-venv on your Ubuntu system.
      • Command: apt install python3-pip python3-venv (Ensure that you have the priviledge to install packages, if not, please use Super User.)

Create a new directory for OpenMetadata and navigate into that directory.

Create a virtual environment to avoid conflicts with other Python environments on your host system. A virtual environment is a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages.

In a later step you will install the openmetadata-ingestion Python module and its dependencies in this virtual environment.

After running the command above, you should see output similar to the following.

This will create a docker network and four containers for the following services:

  • MySQL to store the metadata catalog
  • Elasticsearch to maintain the metadata index which enables you to search the catalog
  • Apache Airflow which OpenMetadata uses for metadata ingestion
  • The OpenMetadata UI and API server

After starting the Docker containers, you should see an output similar to the following.

After starting the containers, metadata will launch Airflow tasks to ingest sample metadata and usage data for you to experiment with. This might take several minutes, depending on your system.

  • metadata docker --stop will stop the Docker containers.
  • metadata docker --clean will clean/prune the containers, volumes, and networks. You will need to run this if you are updating the OpenMetadata version. Note that it will get rid of the data. If you want to keep it, you will need to Backup your data.

From 0.12, OpenMetadata also supports Postgres local deployment out of the box!

You just need to run:

Note that the option -db postgres needs to be passed to the other commands as well to locate the proper compose file.

˚ Once metadata ingestion has finished and the OpenMetadata UI is ready for use, you will see output similar to the following.

The metadata CLI is very useful for quickly testing when getting started or wanting to try out a new release.

If you had already set up a release and are trying to test a new one, you might need to run metadata docker --clean to clean up the whole environment and pick up the new ingredients from a fresh start.

UI

OpenMetadata ships with an Airflow container to run the ingestion workflows that have been deployed via the UI.

In the Airflow, you will also see some sample DAGs that will ingest sample data and serve as an example.

You can access Airflow at http://localhost:8080. Use the following credentials to log in to Airflow.

  • Username: admin
  • Password: admin
tour

OpenMetadata provides a default admin account to login.

You can access OpenMetadata at http://localhost:8585. Use the following credentials to log in to OpenMetadata.

  • Username: admin
  • Password: admin

Once you login, you can goto Settings -> Users to add another user and make them admin as well.

Please follow our Enable Security Guide to configure security for your OpenMetadata installation.

If you want to persist your data, prepare Named Volumes for the containers.

  1. Visit the Features overview page and explore the OpenMetadata UI.
  2. Visit the Connectors documentation to see what services you can integrate with OpenMetadata.
  3. Visit the API documentation and explore the rich set of OpenMetadata APIs.

If you are getting an error such as "compose" is not a docker command, you might need to revisit the installation steps above to make sure that Docker Compose is properly added to your system.

Are you having trouble starting the containers with the metadata CLI? While that process is recommended, you can always run docker compose manually after picking up the latest docker-compose.yml file from the release:

This snippet will create a directory named openmetadata and download the docker-compose.yml file automatically. Afterwards, it will start the containers. If instead you want to download the file manually to another location, you can do so from the Releases page.

This will start all the necessary components locally. You can validate that all containers are up and running with docker ps.

In a few seconds, you should be able to access the OpenMetadata UI at http://localhost:8585:

You might see something like:

A common solution is to run docker network prune:

So be careful if you want to keep up some (unused) networks from your laptop.

You can connect Docker containers to communicate with Host Operating System Services. Navigate to the official docker documentation which will help achieve the same.

If you're running the metadata docker --start after updating the openmetadata-ingestion package to a newer OpenMetadata release, you might encounter issues. For example, java.lang.ClassNotFoundException: org.openmetadata.service.security.NoopAuthorizer when updating from 0.11.

Then, you'll need to first run metadata docker --clean. It will clean/prune the containers, volumes, and networks, and download the newest docker compose file from the release. Note that this will get rid of the data. If you want to keep it, you will need to Backup your data. This command is required after each time you want to update your quickstart deployment of OpenMetadata with a new release to pick up the new compose file.