Introduction

OpenDDL is a standard API for ORMs to export a project's schema metadata for interoperability with other tools.

Introduction

ORMs have become a ubiquitous way of writing software that interacts with databases. By allowing users to model entities and relationships and express queries and operations in native programming languages, they act as a bridge between application environments and their underlying databases.

Although routinely criticized and seen as ineffective in many circumstances by many, ORMs continue to grow as a dominant way of writing modern applications.

Purpose

Today, discovering the underlying database schema of most ORM-based applications requires installing the application on a live database and using schema inspection tools to extract the application's data structure from the database's information schema tables.

As the data landscape continues to evolve and become more complex, ORM-based applications must interoperate with many other kinds of tools in many other areas such as schema management, database compliance, data catalogs, Data Pipelines ( ETL/ELT) and more.

Ultimately, ORM-based data models always translate into a specific database schema. However, as mentioned above, this schema is not easily exposed for consumption by others. The purpose of the OpenDDL API is to provide a standard way for ORMs to export the application's desired database schema

Design Considerations

It seems that our industry is saturated with specifications and metadata formats that attempt to standardize schema definition and description. With OpenDDL, we have decided that instead of creating an additional layer of abstraction that imposes translation and interoperability concerns, we will stay close to the true database schema and utilize each database's data definition language (DDL).

Annotations are added as a method of including additional metadata on schemas that can be used to modify the behavior of downstream consumers. Annotations may, for example, carry information such as the sensitivity of a given entity or field (for example, "contains PII" or "encrypted").

Architecture

Example Output

The following JSON document is a valid OpenDDL Schema document:

{
  "apiVersion": "openddl.io/v1",
  "kind": "Schema",
  "metadata": {
    "dialect": "mysql",
    "annotations": {
      "openddl.io/orm": "gorm"
    }
  },
  "spec": {
    "ddl": "CREATE TABLE auth.users ( id INT, name VARCHAR(100), email VARCHAR(100));",
    "annotations": [
      {
        "selector": {
          "schema": "auth",
          "table": "users",
          "column": "email"
        },
        "properties": {
          "openddl.io/pii": "true",
          "openddl.io/data-type": "email"
        }
      }
    ]
  }
}