situations do threaten the validity and reliability of the assessment.
The quantification part of the audit is still very experimental.
Nevertheless, this step forces the auditors to make
detailed assessments first and then aggregate these to the level
of delivery systems. These global assessments are initially
fed back to the company for response, which often results in
useful comments. This whole chain of steps assures that the
audit team does not jump to premature conclusions having
no validity whatsoever.
During the development of the audit some pragmatic decisions
had to be made that still need some verification. The
issue of whether to use protocols versus diagrams during the
interviews is one. However, an auditor should be quite aware
of what any step within any delivery system entails before
(s)he can make do without the protocols. Obviously, an audit
such as the current one is quite demanding and would require
auditors that are well trained in its philosophy. Both the standardisation
of the approach and the training of auditors in it
will heighten the reliability of the audit results.
In addition to the important issues of validity and reliability,
there are still other matters open for questioning. For
instance, the weighting of the steps within the delivery systems
remains to be explored more extensively. Although the
reasoning of equally and unequally weighted systems appears
to be sound, there is now no empirical evidence for its support.
Probably, both an extensive literature survey and expert
judgement are needed to supply more answers.
Lastly, the relationship between barrier types and delivery
systems is still open for research. What (type of) barrier
does benefit from what (type of) management influence? In an
attempt to fill in these gaps a newproject has been formulated
called D-SMART [15] that should provide some answers to
these questions. In the mean time, the audit and the accompanying
tool will be available upon request for anybody’s use
and useful comments.