Skip to main content

Configure Object Storage

OpenMetadata saves the files that people upload, such as the files in Context Center > Documents, in object storage. Object storage is turned off by default. Turn it on before you let people upload files.
If object storage is off, an upload to Documents looks successful and the file appears in the list, but OpenMetadata doesn’t save the file contents. Downloading the file then fails with No current content found for this file. Files uploaded while storage was off can’t be recovered. Turn storage on, then upload them again.

Settings

Add the objectStorage block to openmetadata.yaml, or set the environment variables shown for each setting. Restart OpenMetadata after you change them.

Amazon S3

OpenMetadata finds credentials in this order:
  1. If you set s3.endpoint, it uses s3.accessKey and s3.secretKey.
  2. Otherwise, it uses the standard AWS credential chain, such as an IAM role on the server or pod, or the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.
  3. If the chain finds nothing, it uses s3.accessKey and s3.secretKey.
The credentials need s3:PutObject, s3:GetObject, and s3:DeleteObject on the bucket and prefix.

MinIO and Other S3-Compatible Storage

Use provider s3 and set s3.endpoint (ASSET_UPLOADER_S3_ENDPOINT) to the address of your storage server. OpenMetadata then uses the access key and secret key you configure.

Azure Blob Storage

OpenMetadata signs in with the default Azure credentials of the server or pod, such as a managed identity or the AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_CLIENT_SECRET environment variables. That identity needs permission to read, write, and delete blobs in the container.

Check That It Works

  1. Upload a small file in Context Center > Documents.
  2. Download the same file. It should open normally.
  3. Check that a new object appears in your bucket or container, under the prefix you set.
If the download still fails, check these items:
  • Storage is off or the server wasn’t restarted. Confirm that objectStorage.enabled is true and that you restarted OpenMetadata.
  • The provider is inmemory. Files are lost when the server restarts, and a second server can’t read files saved by the first. Use s3 or azure.
  • The upload failed on the storage side. With s3 or azure, a failed write makes the upload fail with an error, and OpenMetadata removes the new file entry. Read the server log from the moment of the upload for storage errors, such as access denied.
  • The credentials can’t write or read. Check the permissions in Amazon S3 or Azure Blob Storage.
  • The file is too large. Raise objectStorage.maxFileSize.
For more information about Documents, see Documents and Archive.