Document and event indexing

Data from documents and events are added to the index tables automatically as the documents and events are loaded.

For documents, a post processor runs a class defined in the Document Index Script. This creates a denormalised join of index value and document index and submits it to the load/document_index entity. This loads the index values first, then the document index.

The Document Index class has a buildIndex() method which is passed a where clause for a Data Hub query that extracts the document and associated worker. When called as part of the document load, the where clause is set to exactly true, which selects the document or documents that have just been loaded.  The data returned from the query is broken down into index values and written as a new message to the store.

Events follow the same pattern as documents, using the Event Index Script.