Doing Assessment and Improvement

Intro to assessment, staged models vs risk-based

Intro to contents of next level

Intro to assessment

The model-based method of Capability Evaluation (CE) has three styles of use.

Project monitoring, and specifically monitoring of Process Improvement (PI).

Internal assessment of its own processes by an enterprise (or part of an enterprise).

Contractor selection in major projects where quality in use has been identified as a risk area - Pre Contract Award Evaluation (PCAE).

The scope of an assessment (in terms of processes assessed) is tailored to the needs of the project. Similarly, the size of an assessment (in terms of organisations assessed) can be tailored to circumstances. It is possible to conduct very low effort assessments at the early stage of a project, on small projects and for informal process risk/PI purposes.

Process standards are principally concerned with two sorts of use, process improvement (PI), and capability evaluation (CE). However, they have proved to have other uses e.g. planning, competence definition. PI is typically conducted by an organisation for its own benefit (but can be mandated by a customer) and comprises a cycle of self-assessment, structured improvement and re-assessment. CE can vary in formality, but at its most formal is done by an accredited third party for the purposes of pre-contract award vendor assessment. The systems community has developed COTS tools and methods for both PI and CE, one of the many benefits from adopting the process standard approach.

The diagram below shows how project risks and project requirements provide the inputs to a Process Risk Assessment, which is followed by Process Improvement action and tracking.

Process assessment, capability and maturity

A disciplined evaluation of an organisation’s processes against a model is called process assessment. Process assessments generally focus on identifying improvement priorities (i.e. a formative evaluation). Action taken to change an organisation’s processes so that they meet the organisation’s business needs and achieve its business goals more effectively is called process improvement. Process assessment and improvement are described in more detail below. Process assessment seeks firstly to establish whether processes are performed successfully and secondly the degree to which processes are under control. A scale of capability is used in this assessment.

A capability scale is an ordinal scale of types of control. The table below describes the most common scale. This uses six levels to assess process capability.

Process capability scale

Level

Description

0

No achievement of results from a processes or processes

1

Performance in an ad hoc manner

2

Monitoring of time and product quality

3

Use of defined corporate procedures and infrastructure

4

Use of statistical control

5

Optimisation of each process to meet current and future business needs

Some words on the approach to 'capability' and 'maturity' are appropriate to the reader familiar with the terms from CMM ®. This paper is not the place for a debate on the merits of 'staged' and 'continuous' models. The HS model is a 'continuous' model in accordance with the ISO approach, whereby processes are selected for assessment and the organisation's capability to perform a process is assessed against a capability scale. It can of course be mapped to the stages of a staged model.

A staged model such as CMM offers strong intuitive appeal by presenting a set of organisational maturity levels, which are recognisable plateaux of aggregated process capability. The application of this concept to HCD featured in the research leading to the UMM, in the form of a Human Centredness Scale (Earthy, 1998). This scale portrays levels of human-centredness from X (Unrecognised) through to E (Institutionalised) via A (Recognised), B (Considered), C (Implemented) and D (Integrated).

Human-centred methods developed for organizations with high levels of maturity cannot be adopted by organizations at a lower level of maturity. The supporting processes and information flows necessary to make the method work do not exist and the attitude necessary to put them in place is not widespread enough to make a difference. The scale has been used for a variety of purposes, notably significant evaluations in the Chinese IT sector.

The authors consider that the primary use of the HCS is educational and motivational, rather than formal assessment. A limited process capability approach might be adapted for similar ends using sub-processes from the HS model selected from the strategic end of HS.2, such as HS.2.1, HS.2.2, HS.2.3. More formally, as mentioned above, ISO PAS 18152 has an annex that identifies the sub-processes that require or imply a level of process maturity above level 1.