Usability goals

Tools, devices or software (as diverse as a TV remote control, the interface of an oven, or a word processor) must be evaluated before their release on the market from different points of view such as their technical properties or their usability. Usability evaluation allows assessing whether the product under evaluation is efficient enough (Are the users able to carry out their task while expending reasonable resources such as time, cognitive or physical demand), effective enough (Can the user complete the tasks they are supposed to perform with the tool? Is their performance complete and accurate?) and sufficiently satisfactory for the users (What is the users’ attitude towards the system? Do they experience discomfort?).^[1][2] For this assessment to be objective, there is a need for measurable goals^[3] (for instance in terms of easiness of use or of learning) that the system must achieve. That kind of goal is called a usability goal (or also usability requirement^[1][4]). They are objective criteria against which the results of the usability evaluation are compared to assess the usability of the product under evaluation.^[2]

In product design

Usability goals must be included in every product design process that intends to follow a Human Factors approach (for instance, User-centered design^[1] process or Usability Engineering Lifecycle^[5]). They have to be clearly stated from the onset of the process, as soon as the end-users needs, risk of use, contexts and aims of use are identified (cf. “definition of usability goals” part).

Then, usability goals are used at each usability evaluation phase of the design process. Whatever the type of evaluation phase (i.e. formative or summative evaluation^[6]), they are used to assess the performance of the users against the result of the evaluation process:

• During formative/constructive evaluations (i.e. evaluations that occur during the design process to contribute to further improvement of the object under evaluation^[6]), the comparison of the evaluation results against usability goals allows verifying whether those goals are met or not: as long as they are not met, the product under evaluation must be re-engineered to improve its usability. In this frame, usability goals allow also identifying usability flaws and therefore supporting this re-engineering process. They can also be used all along the iterations of the User-centered design process as indicators to follow up the evolution of the system in terms of usability.
• During summative evaluations (i.e. evaluations that try to give a definitive statement on the quality properties of a system under evaluation^[6]), the meeting of usability goals means that the system is usable enough to go out the User-centered design^[1] process and to be released.

Formulation

Usability goals must address the three usability components, i.e. effectiveness, efficiency and satisfaction.^[2] Their definition, for each of those components, must rest on the characteristics of the tasks that the tested system is supposed to support.^[2] More practically, Mayhew ^[5] proposes that their definition should refer to:

• The identified end-users profiles
• The tasks that the different categories of identified end-users are supposed to perform with the tested system in a given context of use (results from a Contextual Task Analysis).
• Business goals

Moreover, for certain types of products that are used for sensitive purposes (for instance, medical devices or nuclear plant control interface), usability goals must be defined in close relation to the Risk assessment process of those products.^[7][8] This kind of “safety-oriented usability goal” is used to prevent a tool being released on the market while identifying deficiencies in its interface design that could induce Use errors. Thus, risks that may result in use errors must be identified; and then, for each of them, usability goals must be defined, taking into account the severity of the potential consequences of the risk^[4][9](for instance, in terms of operator, patient or environment safety).

Prioritization

For a given tool under evaluation, several usability goals are defined. If some goals are related to safety issues while others are more “comfort of use usability goals", they will not all require the same level of achievement.

For instance, a “comfort of use usability goal” dealing with the easiness of browsing on the Internet that does not endanger users' safety could require a partial achievement (e.g. 80% of users must achieve using a function that make easier the browsing on the Internet, as a short-cut) while a usability goal concerning a major risk for users' or environment' safety would require a total achievement (no error tolerated; e.g.100% of the users must succeed in using a defibrillator at their first trial). For this kind of “safety-oriented usability goal”, a non-achievement reveals that the use of the tool may lead to dramatic consequences. Those goals should be satisfied before any release of the system (for instance, a patient safety sensitive Health Information Technology cannot be released if it has been shown to induce errors of use ^[7][8]).

Therefore, the achievement level of the defined usability goals should be prioritized.^[5]

Measurement

The goals are defined either in a qualitative or a quantitative way.^[5] Nonetheless, whatever their nature, they have to be operationally defined. The achievement of qualitative usability goals can be assessed through verbal protocols analysis. Then, the goal will be formulated in terms related to the coding scheme used for the analysis. Those qualitative goals can be turned into quantitative goals to support an objective quantifiable assessment. This kind of goal can take the shape of:

• "U% of a sample of the intended user population should express positive comments about a specific function while using the tool"
• or “less than U% of the sample misinterprets the information provided by a display”.

As for qualitative usability goals assessed through questionnaires, they can be formulated as:

• “The average score of the sample of the intended user population for the scale S must be over N”

As for quantitative goal, they can be assessed by various methods such as time measurement (instance in ^[2]), keystroke analysis or error rate quantification. They may look like (following^[3][10]):

• “U% of a sample of the intended user population should accomplish T% of the benchmark tasks within M minutes and with no more than E errors”

See also

References

1. 1 2 3 4 International Organization for Standardization. Ergonomics of human system interaction - Part 210 -: Human centred design for interactive systems (Rep N°9241-210). 2010, International Organization for Standardization
2. 1 2 3 4 5 Nielsen, Usability Engineering, 1994
3. 1 2 Salvemini, A. V. (1999). "Challenges for user-interface designers of telemedicine systems". Telemedicine journal. 5 (2): 163–168. PMID 10908428. doi:10.1089/107830299312122. 
4. 1 2 van der Peijl, Jorien; Klein, Jan; Grass, Christian; Freudenthal, Adinda (August 2012). "Design for risk control: the role of usability engineering in the management of use-related risks". Journal of Biomedical Informatics. 45 (4): 795–812. doi:10.1016/j.jbi.2012.03.006. 
5. 1 2 3 4 Mayhew. The usability engineering lifecycle: a practitioner's handbook for user interface design. London, Academic press; 1999
6. 1 2 3 Brender J. Handbook of evaluation methods for health informatics. Burlington, MA: Elsevier; 2006.
7. 1 2 Schertz, JC; Saunders, H; Hecker, C; Lang, B; Arriagada, P (September 2011). "The redesigned follitropin alfa pen injector: results of the patient and nurse human factors usability testing". Expert Opinion on Drug Delivery. 8 (9): 1111–1120. PMID 21843107. doi:10.1517/17425247.2011.608350. 
8. 1 2 Marcilly et al., Patient Safety Oriented Usability Goals: a pilot study. MIE 2013.
9. ↑ Association for the Advancement of Medical Instrumentation. Human factors engineering-design of medical devices (ANSI/AAMI HE75). Arlington, VA: AAMI; 2009.
10. ↑ Smith E. Siochi A. Software usability: requirements by evaluation. In: Human factors perspectives on human-computer interaction. Santa Monica, CA: Human factors and Ergonomics Society, 1995.