Data Governance Portal

Data Governance Portal

The Data Governance Portal provides a user interface to define data governance scopes based on repository filters and tracked fields.
These scopes are used to define which fields will be audited regarding its completeness and currentness.
E.g. ‘Business Applications that are productive and critical need to have these fields maintained and reviewed in this interval'.

To open the portal navigate to Data Governance -> Data Governance Portal and open the portal to start creating data governance scopes.

A – Indicates the amount of scopes ( 1 ), the amount of fields tracked [ 1 ] and table 'Symbol + Table Name'. Hover over the icon to check if it is a scope or field.

B – These ‘cards’ represent a Data Governance Scope

C – Click here to create a new Scope for the selected/expanded table

D – Click here to create a new Scope for a new table not being tracked yet

Creating a new Scope: step-by-step guide

A – Click the plus sign in the center of a card within a table definition to create a new scope for that table or the plus sign in the bottom right to create a new scope for a new table

B – Provide a meaningful name, e.g. ‘Bus. Applications in Production’

C – These buttons are used for saving or discarding the scope

D – Specify the target table

E – Specify the steward type – steward type can be assigned to a person or group. If a the type is assigned to a single person, they must be referenced from the user-table.

F – Review cycle specifies how frequently the data's validity will be reset. Once reset the data must be again verified (as current)

G – Reminder cycle determines the interval people are reminded to verify. The interval begins from the day the scope is created.

H – You can toggle the audit option off if the field is not required for data completeness and currentness reviews. If audit is active, the steward, or steward group members, will receive notifications directly. Additionally, completeness and currentness will appear in the form view. You can also switch off the notifications and check the form view only for status updates.

I – Select the checkbox to include all records in the table or click on the ‘Filters’ button to open a window to specify filter criteria for the scope records to satisfy

J – Specify your filter criteria here and confirm with ‘Save’

K – After the scope card has been created, you can add fields relevant to the scope/to monitor

🚧

Caution when working with huge amounts of data

When creating scopes for Data Governance, UPM-X creates evaluation entries in the data quality table (x_inpgh_upmx_data_quality) for each tuple of table and field.
For tables with large amounts of data (>10.000 elements), we recommend creating the scope with 2-3 fields, waiting for the evaluation to complete, and adding the next set of fields.

Data Quality Widget

The Data Quality Widget provides an overview of tracked fields and allows to easily verify fields that are due for verification.
Control its visibility with ACLs on the Data Quality tab of an element/table where data quality is tracked.

A – Field tracked by Data Governance

B – Current value of the field or ‘(Empty)’

C – Completeness Indicator – either ‘Complete’ or ‘Missing’

D – Currentness Indicator – either ‘Current’, ‘Overdue’ or ‘Missing’/’No Value’

E – ‘Next verification in’ indicates when the next review is due according to the review cycle definition, or how many days/months/years it is overdue

F – Verification button to instantly verify the field content

G – ‘Verify All’ – instantly verifies all fields where a verification is due

H – Example of countdown to next due verification

Data Governance Reporting

UPM-X stores Data Governance data in these tables

  • x_inpgh_upmx_data_quality - main data quality table
  • x_inpgh_upmx_portfolio_element - all UPM-X tables are extended from this and have a calculated ‘Data Completeness’ and ‘Data Currentness’ field
  • Since v4.2 calculated ‘Data Completeness’ and ‘Data Currentness’ values are available on all tables inherited from ‘cmdb_ci’

To create Data Governance Reports and Dashboards, start with the Data Quality table and if you need to filter on further criteria on the tables themselves, switch to the attributes directly on the tables.

Scheduled Jobs in Data Governance

The Data Governance feature in UPM-X leverages Scheduled Jobs, especially to conduct administrative work

A – Sets data currentness to zero on Data Quality table if it exceeds the review cycle

B – Sends the notification containing information about Data Completeness and Currentness to steward/steward group if reminder cycle is over

C – Sets the ‘Value’ field on Data Quality table with the actual field value

D – Sets the average data completeness and currentness for an element in ‘Data Completeness’ and ‘Data Currentness’ fields of the element

E – Removes the entire Data Quality configuration – e.g. in case you want to get rid of demo data/configuration and start with a clean slate

F – Creates Data Quality records for the scopes defined

G – Sets the Steward Type, Reminder Cycle and Last notified Date fields on Data Definition Scope  ‘Legacy/pre-v4.1’ for migration purposes

🚧

Caution when clearing data quality data

Use ‘Remove complete Data Quality configuration’ with caution – it removes everything related to DG. It might be an alternative to clear the x_inpgh_upmx_data_quality table to achieve the desired outcome

Notifications in Data Quality

UPM-X leverages the ServiceNow platform functionality for sending notifications to stakeholders. Notifications are sent out for scopes defined in the data governance portal that are flagged with 'audit=yes'.

  • To edit Notifications navigate to ‘System Notification -> Notifications’
  • To edit Email scripts, navigate to ‘System Notification -> Notification Email Scripts’

A – List of notifications in the context of UPM-X Data Governance. The notification invokes a corresponding Notification Email Script

B – Name of the Notification

C – Name of the Notification Email Script invoked

D – List of Notification Email Scripts in the context of UPM-X Data Governance.

📘

Set the notification schedule via data governance portal

The schedule when the notifications are sent out is controlled by the Data Governance Portal settings in the individual scopes