Adding Custom Tests

Publish results from an external quality tool or implement a validator that OpenMetadata runs. Steps 1–4 register a custom test and publish its results through the API. Step 5 describes installing a custom validator for execution through OpenMetadata. Before starting, ingest the target table into OpenMetadata. Obtain a bearer token with permission to create test definitions, edit tests on the table, and publish results. Replace {access_token} in the examples and use your server URL instead of http://localhost:8585. The examples use the existing table local_redshift.dev.dbt_jaffle.customers. Replace this fully qualified name (FQN) throughout with your table’s FQN.

Step 1: Create a Test Definition

Send POST /api/v1/dataQuality/testDefinitions to register a definition. This example declares a string parameter named colName for a test whose results are calculated externally.

curl --request POST 'http://localhost:8585/api/v1/dataQuality/testDefinitions' \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "description": "A demo custom test",
    "entityType": "TABLE",
    "name": "demo_test_definition",
    "testPlatforms": ["Other"],
    "parameterDefinition": [{
        "name": "colName",
        "dataType": "STRING",
        "required": true
    }]
}'

Keep the fullyQualifiedName from the response (demo_test_definition in this example). Test-case creation accepts this name as a string, rather than an ID or an entity-reference object. To make the definition available for execution through OpenMetadata, include OpenMetadata in testPlatforms and install a matching validator as described in Step 5. Registering a definition alone doesn’t implement or run the test.

Step 2: Create a Test Suite

Optional: Create a suite explicitly before creating a test case. A basic test suite holds the tests for a table. When you create a test case in Step 3, the server reuses the table’s basic suite or creates one automatically. Skip this step unless you want to create the suite explicitly. To create it explicitly, send POST /api/v1/dataQuality/testSuites/basic with the existing table’s FQN in basicEntityReference. Reuse the suite if the table already has one.

curl --request POST 'http://localhost:8585/api/v1/dataQuality/testSuites/basic' \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "name": "local_redshift.dev.dbt_jaffle.customers.testSuite",
    "description": "Custom tests for the customers table",
    "basicEntityReference": "local_redshift.dev.dbt_jaffle.customers"
}'

POST /api/v1/dataQuality/testSuites creates a logical grouping instead. Omit executable and basic from the request. The endpoint determines the suite type. Use basicEntityReference instead of the deprecated executableEntityReference field. For more information, see Create a Test Suite.

Step 3: Create a Test Case

Send POST /api/v1/dataQuality/testCases with the test definition’s FQN as a string and an entityLink identifying the table or column. For a table, use <#E::table::tableFQN>. For a column, use <#E::table::tableFQN::columns::columnName>. Include the opening and closing angle brackets. Omit testSuite from the request. The server associates the case with the table’s basic suite using entityLink. Parameter names must match the definition, and parameter values must be strings.

curl --request POST 'http://localhost:8585/api/v1/dataQuality/testCases' \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "entityLink": "<#E::table::local_redshift.dev.dbt_jaffle.customers>",
    "name": "custom_test_Case",
    "testDefinition": "demo_test_definition",
    "parameterValues": [{
        "name": "colName",
        "value": "id"
    }]
}'

Keep the returned fullyQualifiedName for Step 4. For this example, it is local_redshift.dev.dbt_jaffle.customers.custom_test_Case.

Step 4: Write Test Case Results

Optional: Publish results yourself when you run the test outside OpenMetadata. After running your external test, send POST /api/v1/dataQuality/testCases/testCaseResults/{fqn} with its result. Use the test case’s returned FQN, URL-encoded when necessary, in the path. The required fields are timestamp, testCaseStatus, and testResultValue. The timestamp is Unix epoch time in milliseconds. Replace the example timestamp with the actual execution time.

curl --request POST 'http://localhost:8585/api/v1/dataQuality/testCases/testCaseResults/local_redshift.dev.dbt_jaffle.customers.custom_test_Case' \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "result": "All 10 checked rows passed the custom validation",
    "testCaseStatus": "Success",
    "timestamp": 1769982800000,
    "testResultValue": [{
        "name": "rowsChecked",
        "value": "10"
    }]
}'

View the result on the table’s test case. For more information, see Test Case Results.

Step 5: Make the Test Available Through the OpenMetadata UI

Optional: To run the test through the OpenMetadata user interface (UI), implement a validator in the data_quality namespace.

1. Create Your Namespace Package

Create a Python package for your validation logic with this minimum structure:

metadata/
setup.py

For SQLAlchemy sources, place your validation file in the directory for its entity type:

Table tests: metadata/data_quality/validations/table/sqlalchemy/<yourTest>.py
Column tests: metadata/data_quality/validations/column/sqlalchemy/<yourTest>.py

<yourTest> should match the name of your test definition in Step 1. Add an __init__.py file in every package folder with this line:

__path__ = __import__('pkgutil').extend_path(__path__, __name__)

2. Create Your Test Class

In <yourTest>.py, create a class named <YourTest>Validator that inherits from BaseTestValidator. Inherit from SQAValidatorMixin if you need its helper methods. Implement run_validation to return a TestCaseResult object. The following illustrative skeleton omits imports and validation logic. Add these in your implementation before running it.

class ColumnEntropyToBeBetweenValidator(BaseTestValidator):
    """Implements custom test validator for OpenMetadata.

    Args:
        BaseTestValidator (_type_): inherits from BaseTestValidator
    """

    def run_validation(self) -> TestCaseResult:
      """Run test validation"""

3. Install Your Package

After implementing the validator, install your package with pip install in the environment that runs the OpenMetadata Python SDK.

Data Quality and Observability

Custom Tests | OpenMetadata Quality Testing Guide

Adding Custom Tests

Step 1: Create a Test Definition

Step 2: Create a Test Suite

Step 3: Create a Test Case

Step 4: Write Test Case Results

Step 5: Make the Test Available Through the OpenMetadata UI

1. Create Your Namespace Package

2. Create Your Test Class

3. Install Your Package

​Adding Custom Tests

​Step 1: Create a Test Definition

​Step 2: Create a Test Suite

​Step 3: Create a Test Case

​Step 4: Write Test Case Results

​Step 5: Make the Test Available Through the OpenMetadata UI

​1. Create Your Namespace Package

​2. Create Your Test Class

​3. Install Your Package

Adding Custom Tests

Step 1: Create a Test Definition

Step 2: Create a Test Suite

Step 3: Create a Test Case

Step 4: Write Test Case Results

Step 5: Make the Test Available Through the OpenMetadata UI

1. Create Your Namespace Package

2. Create Your Test Class

3. Install Your Package