Data Governance Portal

The Data Governance Portal provides a user interface to define data governance scopes based on repository filters and tracked fields.
These scopes are used to define which fields will be audited regarding its completeness and currentness.
E.g. ‘Business Applications that are productive and critical need to have these fields maintained and reviewed in this interval'.

To open the portal navigate to Data Governance -> Data Governance Portal and open the portal to start creating data governance scopes.

A – Indicates the amount of scopes ( 1 ), the amount of fields tracked [ 1 ] and table 'Symbol + Table Name'. Hover over the icon to check if it is a scope or field.

B – These ‘cards’ represent a Data Governance Scope

C – Click here to create a new Scope for the selected/expanded table

D – Click here to create a new Scope for a new table not being tracked yet

Creating a new Scope: step-by-step guide

A – Click the plus sign in the center of a card within a table definition to create a new scope for that table or the plus sign in the bottom right to create a new scope for a new table

B – Provide a meaningful name, e.g. ‘Bus. Applications in Production’

C – These buttons are used for saving or discarding the scope

D – Specify the target table

E – Specify the steward type – steward type can be assigned to a person or group. If a the type is assigned to a single person, they must be referenced from the user-table.

F – Review cycle specifies how frequently the data's validity will be reset. Once reset the data must be again verified (as current)

G – Reminder cycle determines the interval people are reminded to verify. The interval begins from the day the scope is created.

H – You can toggle the audit option off if the field is not required for data completeness and currentness reviews. If audit is active, the steward, or steward group members, will receive notifications directly. Additionally, completeness and currentness will appear in the form view. You can also switch off the notifications and check the form view only for status updates.

I – Select the checkbox to include all records in the table or click on the ‘Filters’ button to open a window to specify filter criteria for the scope records to satisfy

J – Specify your filter criteria here and confirm with ‘Save’

K – After the scope card has been created, you can add fields relevant to the scope/to monitor

🚧
Caution when working with huge amounts of data
When creating scopes for Data Governance, UPM-X creates evaluation entries in the data quality table (x_inpgh_upmx_data_quality) for each tuple of table and field.
For tables with large amounts of data (>10.000 elements), we recommend creating the scope with 2-3 fields, waiting for the evaluation to complete, and adding the next set of fields.

Data Quality Widget

The Data Quality Widget provides an overview of tracked fields and allows to easily verify fields that are due for verification.
Control its visibility with ACLs on the Data Quality tab of an element/table where data quality is tracked.

A – Field tracked by Data Governance

B – Current value of the field or ‘(Empty)’

C – Completeness Indicator – either ‘Complete’ or ‘Missing’

D – Currentness Indicator – either ‘Current’, ‘Overdue’ or ‘Missing’/’No Value’

E – ‘Next verification in’ indicates when the next review is due according to the review cycle definition, or how many days/months/years it is overdue

F – Verification button to instantly verify the field content

G – ‘Verify All’ – instantly verifies all fields where a verification is due

H – Example of countdown to next due verification

Data Governance Reporting

UPM-X stores Data Governance data in these tables

x_inpgh_upmx_data_quality - main data quality table
x_inpgh_upmx_portfolio_element - all UPM-X tables are extended from this and have a calculated ‘Data Completeness’ and ‘Data Currentness’ field
Since v4.2 calculated ‘Data Completeness’ and ‘Data Currentness’ values are available on all tables inherited from ‘cmdb_ci’

To create Data Governance Reports and Dashboards, start with the Data Quality table and if you need to filter on further criteria on the tables themselves, switch to the attributes directly on the tables.

Scheduled Jobs in Data Governance

The Data Governance feature in UPM-X leverages Scheduled Jobs, mainly to conduct administrative work. The following scripts are available for use on scheduled jobs.

These have the following actions

Name	Function
IGNORE: UPM-X DO Create Data Quality for CIs created in last 15 minutes	DEPRECATED
ON DEMAND: UPM-X DQ Add missing data in Data Definition Scope table	Sets the Steward Type, Reminder Cycle and Last notified Date fields on Data Definition Scope  ‘Legacy/pre-v4.1’ for migration purposes
ON DEMAND: UPM•X DQ Create Data Quality from all Scoped Fields	Recreates content of the Data Quality table based on the definition/configuration
ON DEMAND: UPM-X DQ Fill Table Reference field in Data Quality	Forces update for the table_reference in the Data Quality table for all records
ON DEMAND: UPM-X DQ Remove complete Data Quality configuration	Removes the entire Data Quality configuration – e.g. in case you want to get rid of demo data/configuration and start with a clean slate
ON DEMAND: UPM-X DQ Set 'Verification History' on Data Quality from 'Data Verification' table	DEPRECATED Takes the last verified day from the Data Verification table to the Data Quality table
ON DEMAND: UPM-X DQ Set Average Data Quality.	Sets the average data completeness and currentness for an element in ‘Data Completeness’ and ‘Data Currentness’ fields of the element
PERIODIC: UPM-X DQ Create Data Quality from recently created Scoped Fields	Calculates Data Quality values after the scope is created.
PERIODIC: UPM-X DO Update Data Quality for recently created/updated CIs	Re-calculates Data Quality values for recently updated records.
PERIODIC: UPM-X DQ Update Data Quality when Scope changes	Calculates Data Quality values after the scope is updated.
UPM-X DQ Notify Currentness and Completeness info to Steward/Steward Group	Sends the notification containing information about Data Completeness and Currentness to a steward/steward group if the reminder cycle is overdue
UPM-X DQ Set Data Currentness on Data Quality table	Sets data currentness to zero on Data Quality table when the previous cycle becomes overdue

🚧
Caution when clearing data quality data
Use ‘Remove complete Data Quality configuration’ with caution – it removes everything related to DG. It might be an alternative to clear the x_inpgh_upmx_data_quality table to achieve the desired outcome

❗️
If an ON DEMAND feature is not already marked depricated it may be depricated in future.

Notifications in Data Quality

UPM-X leverages the ServiceNow platform functionality for sending notifications to stakeholders. Notifications are sent out for scopes defined in the data governance portal that are flagged with 'audit=yes'.

To edit Notifications navigate to ‘System Notification -> Notifications’
To edit Email scripts, navigate to ‘System Notification -> Notification Email Scripts’

A – List of notifications in the context of UPM-X Data Governance. The notification invokes a corresponding Notification Email Script

B – Name of the Notification

C – Name of the Notification Email Script invoked

D – List of Notification Email Scripts in the context of UPM-X Data Governance.

📘
Set the notification schedule via data governance portal
The schedule when the notifications are sent out is controlled by the Data Governance Portal settings in the individual scopes

Data Governance Portal