IntegrityCheck

In modelling, especially when complexity rises, errors are easily made.

The GeoDMS contains ways of assisting modellers in tracking down and solving errors, think for instance of unit metric consistency.

Another useful feature is the configuration of integrity checks for data items.

An IntegrityCheck is used to check if an (intermediate) result meets certain requirements, for instance that all values need to be within a certain range or that no missing data may occur.

If an IntegrityCheck fails, the data of the resulting items can usually still be requested in a table view, in order to find out what is wrong. The data can not be exported.

A check guards the items below it

Since GeoDMS 20.14.0 an IntegrityCheck is enforced for every item below the item carrying it: requesting a nested item also validates the checks of its ancestors, and a failing ancestor check fails the requested item. In older versions such a check only fired when the checked item was itself requested; reading an item underneath it delivered the data without any validation. A configuration that relied on that silence can start failing on the first run after upgrading — which is the intended effect: the check was configured to gate exactly that data. A typical example is the version guard shown under Recent version below, which now actually protects every item of the configuration.

What counts as "below" is tree position, not item kind. A container and a domain unit are the usual carriers, but an attribute can have sub-items too — a total broken up into its components, or a categorical attribute with further categorisations underneath it — and a check on such an attribute guards those sub-items in exactly the same way.

Two rules to keep in mind when writing such a check:

• The check may not refer to items inside the subtree it guards: that closes a cycle through the guarded items and is reported as a circular dependency. Let it refer to items outside the guarded subtree, for instance a parameter configured next to it. This is what makes the break-up case awkward: total with sub-items comp1 and comp2 cannot carry IntegrityCheck = "total == total/comp1 + total/comp2", because the components would have to be calculated in order to check them and carry that same check themselves. Since 20.14.0 this is refused whichever item is requested, where before it was refused only when total itself was requested. Moving it up to the parent does not help — that parent contains the same items. Express such a cross-item assertion as a separate boolean item that carries the check on itself:

  parameter<bool> total_adds_up := total == total/comp1 + total/comp2
  ,	IntegrityCheck = "total_adds_up == True";
  

  The boolean has no sub-items of its own, and referring to the item that carries the check is the supported self-reference, so nothing reaches into a guarded subtree.

• The condition is calculated once, not once per guarded item, and each guard is enforced once per calculation, no matter how often the guarded items reference each other. When the check fails, the failure is reported on the item(s) directly under the guard; items computed from those fail because their source failed, and their failure reason names that source. So when tracking down a "Validation (Integrity Check) Failed" error, follow the named source item to the check that actually fired.

• The checks travel with the calculation rather than beside it: an ancestor's condition is folded into the calculation of the items below it, so it is scheduled and computed like any other supplier, once, and its verdict reaches every consumer of the guarded item.

If you write an expression in the IntegrityCheck, you can use 'this' to refer to itself. This is preferred.

container ReadOpbrengsten_perOP  := 
	for_each_neidv(OP/name
		, 'ReadOpbrengsten_perOP_UNCHECKED/'+OP/name
		, '(all(IsNull(this)))'
		, AdminDomain
		, EUR
	);

examples

No missing data

attribute<city> city_rel (neighborhood) := rlookup(city_code, regions/city/city_code)
, IntegrityCheck = "isDefined(city_rel)";

The attribute uses the rlookup function to find the relation to the regions/city domain unit.
The configured IntegrityCheck checks if for each neighborhood a city is found.

In such an IntegrityCheck it is allowed to refer to the item for which the IntegrityCheck is configured, the GeoDMS has a workaround to work with such self references.

Recent version

container root: IntegrityCheck = "GeoDmsVersion() >= 7.123"

This IntegrityCheck at the root container of the configuration checks if the configuration is opened in a GeoDMS version 7.123 or later.

Since 20.14.0 this guard actually protects the whole configuration: every item under the root carries it, so an older version is refused whatever item is requested. Before 20.14.0 it only fired when the root container itself was requested, which a model run never does.

not on an item stored in an MMD

An IntegrityCheck can not be configured on an item that is written to an MMD storage: writing such an item is refused, naming the storage holder to configure the check on instead. See MMD for what the storage records about the units its attributes refer to.

see also

• an integrity check can also be used in a for_each loop.