What is the value of Usability? Jan Gulliksen KTH Royal Institute of Technology, Sweden

What is the value of Usability? Jan Gulliksen KTH Royal Institute of Technology, Sweden

Jan Gulliksen Gulan Professor in Human Computer Interaction, Dean of the school Computer Science and Communication, KTH Royal Institute of Technology, Stockholm, Sweden Guest professor at Uppsala university, Sweden Chairman of the Digital commission of the Swedish Government, Ministry of Enterprise Digital champion of Sweden appointed by the European Commission Visiting professor at INSEAD, France

KTH Skolan för Datavetenskap och Kommunikation (CSC)

The Committee for Digitization 2012-2015 Sweden shall be the best country in the world when it comes to using the opportunities for digitization Minister of IT and energy Anna- Karin Hatt and the chairman of the committee Jan Gulliksen ICT for Everyone - A Digital Agenda for Sweden, issued December 2, 2011 First report from the committee was released May 3, 2013

Digital champion of Sweden a high-profile, dynamic and energetic individual responsible for getting everyone in their country online and improving digital skills. You re the digital champion of Sweden Your task is easy Neelie Kroes, vice president European commission

Digital agenda for Sweden Need for achievements within 4 strategic areas based on a user perspective 1. Easy and safe to use, 2. Services that creates some utility, 3. The need for infrastructure, and 4. IT s role for societal development

Usability according to ISO 9241-11 the extent to which a product can be used, by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use ISO 9241-11 Ergonomic requirement of office work with visual display terminals, part 11: Guidance on usability Usability = f(users, tasks, product, context) Task accomplishment is very important Measurements of compliance e.g. according to CIF But are all products now usable? Did it have an impact in practice?

Measures of usability in ISO 9241-11 Usability objective Effectiveness Efficiency Satisfaction Overall usability Percentage of goals achieved. Percentage of users successfully completing task. Average accuracy of completed tasks. Time to complete a task. Tasks completed per unit time. Monetary cost of performing the task. Rating scale for satisfaction. Frequency of discretionary use. Frequency of complaints.

Metrics described in ISO 9241-11 Usability objective Effectiveness Efficiency Satisfaction Meets needs of trained users Number of power tasks performed. Percentage of relevant functions used. Relative efficiency compared with an expert user. Rating scale for satisfaction with power features. Meets needs to walk up and use Percentage of tasks completed successfully on first attempt. Time taken on first attempt 1. Relative efficiency on first attempt. Rate of voluntary use. Meets needs for infrequent or intermittent use. Percentage of tasks completed successfully after a specified period of non-use. Time spent on re-learning functions 1. Number of persistent errors. Frequency of reuse. Minimization of support requirements. Number of references to documentation. Number of calls to support. Productive time 1. Time to learn to criterion 1. Rating scale for satisfaction with support facilities. Number of accesses to help. Learnability Number of functions learned. Percentage of users who manage to learn criterion. Time to learn to criterion 1. Time to re-learn to criterion 1. Relative efficiency while learning. Rating scale for ease of learning. Error tolerance Percentage of errors corrected or reported by the system. Time spent on correcting errors. Rating scale for error handling. Number of user errors tolerated. Legibility Percentage of words read correctly at normal viewing distance Time to correctly read a specified number of characters. Rating scale for visual discomfort. 1. In these examples the resources should be measured in relation to a specified level of effectiveness.

Levels of usability goals Current level. Measured either in the manual process or with the current or competitive product and used as a benchmark to set minimum acceptable levels for the planned product. Minimum acceptable level. Used during iterative evaluation and re-design to determine when to stop iterating. Target level. Used to drive and focus the design effort, actual expected level. Optimal level. Used as a target for the long term. What should be possible if time, money, etc. were not considered? Whiteside, Bennett & Holzblatt, 1988

Why measuring everything? When you can measure what you are speaking about and express it in numbers, you know something about it. But when you cannot measure it, when you cannot express it in numbers, your knowledge is of an unsatisfactory kind: It may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of Science. (Lord Kelvin, 1891)

ISO 9241-210 Human-centered design process for interactive systems

User experiences (UX) "a person's perceptions and responses that result from the use or anticipated use of a product, system or service ISO IS 9241-210 (2010). Ergonomics of human system interaction - Part 210: Humancentered design process for interactive systems (formerly known as 13407). International Organization for Standardization (ISO). Switzerland.

ISO 9241-171 Guidance on software accessibility From usability to accessibility managing equitable use regardless of needs and requirements

Arbetsmiljöns beskaffenhet (Kap.2, 1) Arbetsmiljön skall vara tillfredsställande med hänsyn till arbetets natur och den sociala och tekniska utvecklingen i samhället Arbetsförhållandena skall anpassas till människors olika förutsättningar i fysiskt och psykiskt avseende. Arbetstagaren skall ges möjlighet att medverka i utformningen av sin egen arbetssituation samt i förändrings- och utvecklingsarbete som rör hans eget arbete. Teknik, arbetsorganisation och arbetsinnehåll skall utformas så att arbets-tagaren inte utsätts för fysiska eller psykiska belastningar som kan medföra ohälsa eller olycksfall. Därvid skall även löneformer och förläggning av arbetstider beaktas. Starkt styrt eller bundet arbete skall undvikas eller begränsas. Det skall eftersträvas att arbetet ger möjlighet till variation, social kontakt och samarbete samt sammanhang mellan enskildas arbetsuppgifter. Det skall vidare eftersträvas att arbetsförhållandena ger möjlighet till personlig och yrkesmässig utveckling liksom till självbestämmande och yrkesmässigt ansvar.

Arbetsmiljöns beskaffenhet (Kap.2, 1) Swedish Work Environment Law (Ch.2, 1) Arbetsmiljön skall vara tillfredsställande med hänsyn till arbetets natur och den sociala och tekniska utvecklingen i samhället Arbetsförhållandena skall anpassas till människors olika förutsättningar i fysiskt och psykiskt avseende. Arbetstagaren skall ges möjlighet att medverka i utformningen av sin egen arbetssituation samt i förändrings- The och worker utvecklingsarbete should given som the rör hans eget arbete. possibility to participate in the design Teknik, arbetsorganisation och arbetsinnehåll of skall his/her utformas own så work att arbets-tagaren situation and inte in utsätts för fysiska eller psykiska belastningar changes som kan and medföra development ohälsa eller that olycksfall. concerns the work. Därvid skall även löneformer och förläggning av arbetstider beaktas. Starkt styrt eller bundet arbete skall undvikas eller begränsas. Det skall eftersträvas att arbetet ger möjlighet till variation, social kontakt och samarbete samt sammanhang mellan enskildas arbetsuppgifter. Det skall vidare eftersträvas att arbetsförhållandena ger möjlighet till personlig och yrkesmässig utveckling liksom till självbestämmande och yrkesmässigt ansvar.

User involvement is central In US 250 billion dollars is every year spent on 175 000 different IT-projects. 365 IT-companies with 8380 different IT-projects were analyzed in 1995. 31,1 % of the companies projects were cancelled. 52,7 % were performed with changed plans. 16,2 % were performed according to plan. On average the costs for the changing plans increased with 189 %. 81 billion dollars is every year spent on projects that never leads to any results. CHAOS report, Standish Group, 1995 (www.standishgroup.com)

CHAOS 1994-2010 2010 21% 42% 37% 2008 24% 44% 32% 2006 19% 46% 35% 2004 18% 53% 29% 2002 15% 51% 34% 2000 23% 49% 28% 1998 28% 46% 26% 1996 40% 33% 27% 1994 31% 53% 16% Failed Challenged Succeeded Conclusion: IT development is still a high risk business

McNamara Fallacy The first step is to measure whatever can easily be measured. This is OK as far as it goes. The second step is to disregard that which can't be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can't be measured easily really isn't important. This is blindness. The fourth step is to say that what can't be easily measured really doesn't exist. This is suicide.

86.6% of all statistics are made up on the spot Jacob Nielsen