Data hub target tables

Datahub documentation home

The target entities are represented by database tables. Each table is capable of holding multiple versions of the same record, though in many cases there is only a single version.

As well as the defined business fields, each table has additional fields managed by the hub. These are split into two groups: fields that precede the business fields because they are always relevant, and fields that follow the business fields because they are only used for processing.

Field
Placement
Data type
Description
guid
Before
char(36)
Uniquely identifies this instance of the entity.
valid_from_timestamp
Before timestamp
Point in time from which this version of the entity is valid. Forms the primary key along with the GUID.
valid_to_timestamp
Before timestamp
Point in time up to (but not including) which this row is valid. This is set to null on the current row.
deleted_indicator
Before boolean
Set to true to indicate that the row has been logically deleted.
source_message
After char(36)
The GUID of the message used to create this version of the row.
field_provenance
After text
A JSON object that for each field has an object that identifies the GUID of the message that last updated that field and the priority of that field update.

DDL for MySQL or MariaDB.

CREATE TABLE entity_name (
  guid char(36) NOT NULL,
  valid_from_timestamp timestamp NOT NULL,
valid_to_timestamp timestamp,
deleted_indicator boolean default false,
... business fields ...
source_message char(36),
field_provenance text,
PRIMARY KEY (guid, valid_from_timetamp)
... business keys ...
) ENGINE=InnoDB DEFAULT CHARSET=utf8;