Qualitative User Research Methods. Mattias Arvola

Transkript

1 Qualitative User Research Methods Mattias Arvola

2 2 The process has to begin with some kind of immersion in context and exposure to people who are perhaps experiencing that aspect of life right now. It starts by finding out where we are gonna go, who we are going to speak to, what is their world and really stepping into their shoes. For us to start a design project without that is a bit like cutting of our arms and legs and probably our head as well, we don't know where to start in that case. Anonymous User Experience Designer

3 3

4 4 Visit your users. Speak with them. Write down what you have seen and heard.

5 5 Cost for Change Requests Time

6 6 Cost for Change Requests Impact of UX Time

7 7 Value is created in use by your users. Knowing them, their goals and activities is vital for success.

8 8 Aims of User Research Gain insights on: What we should to design/build/develop Why? How to do it?

9 9 Research Questions Qualitative What? How? Why? Quantitative How much? How long? To what degree? What affect does X have on Y?

10 Qualitative Research 10

11 11 Interpretative research Research is interpretative when the understanding of human sense making is gained through the study of social constructions (e.g. language, shared meanings, consciousness, artefacts etc.) as a situation emerges.

12 12 The Hermeneutical Circle Whole Parts

13 13 Characteristics of Qualitative Research No sharp boundaries of study objects Careful contextualization of the instance Transferability rather than generalizability Cooperation between researcher and participants Purposeful sampling Alternative interpretations Suspicion of interpretations Going beyond what is said and done to what is intended and motivated

14 14 Validation Strategies Prolonged engagement and observation Building trust Learning the culture Checking misinformation and distortions Finding the focus Triangulation Researcher Theory Method Peer review, debriefing and external audits Clarifying what subjective things that have likely shaped the the interpretation past experiences, biases prejudices orientations Member checking Rich and thick description Revise working hypothesis Negative cases with disconfirming evidence

15 Preparations 15

16 16 Initial Framing What is the project about? Finding the target group Not the most representative, but the most important users Who s needs are the most important for us to succeed? Who might have something interesting to say? Who are the ones most affected by the project? Prioritise

17 17 Sensitizing Persona hypothesis Imagine mainstream and extreme users What do they do? In what situations? How do they do it? Why do they do it? What characterizes them in terms of motivations, knowledge and skills. What do they lova an hate?

18 18 How many should we meet? Breadth and depth vs. cost Three participants from three places/contexts As different as possible but within the market segment Recruiting takes time

19 19 Ethics and Informed Consent Knowing that you are observed Understanding what it means Potential risks Gains Anonymity or not Permission to record Provide the participants with the results Introduction in writing

20 Gathering Data 20

21 21 Setting a Focus Getting information and inspiration for design that gives leverage Start with the brief and discuss it with clients Find things to handle in the project

22 22 The Persona and Scenario Sheet A checklist with themes to cover The scenario part: What people do The persona part: How people are Who does what, where, when, how, and why (5WH) Download from:

23 Foundation for personas Goals and driving forces Skills and knowledge Experiences Quote Portrait Family and contacts Likes Name: Description: Dislikes Age: Role: Sex: Habits Income: Hometown: Background Project: Date: Researcher: Persona and scenario sheet. Mattias Arvola 2014.

24 An ordinary day for the persona An ordinary day starts with Project: Date: Researcher: Persona and scenario sheet. Mattias Arvola 2014.

25 Foundation for scenarios OBSERVATIONS AND FACTS WHAT is done? INTERPRETATIONS Weaknesses to address WHY is it done? (participants' motivation and researchers' interpretation) Strengths to emphasize HOW is it done? Constraints to deal with WHO does it and who are involved? WHEN and WHERE is it done? Project: Date: Researcher: Persona and scenario sheet. Mattias Arvola 2014.

26 26 What to look for Observations and facts Opportunities for change Things to preserve and highlight Constraints What characterizes a good/bad situation? Use a protocol for semi-structured interviews and observations (the persona and scenario sheet?) Phenomena and patterns

27 27 Field notes Separate observations and interpretations Use pseudonyms Quotes Sketching with comments and detail studies Note the background, steps and contexts of activities Write the notes up as soon as possible

28 28 Recordings Video, audio or neither Quotes Intrusiveness Transcription and levels of it Clips of episodes No replacement for field notes

29 29 Interviews Open questions that make participants tell stories and explain Tell me about the first time you Tell me about a critical situation when matters were brought to a head Could you describe when you could not Tell me about the last time you Tell me about when it went really well or really bad

30 Keep them talking 30 How do you mean when you say What are you doing now? How do you think about this situation? What are you taking notice of in this situation? Are there any rules of thumb for how to proceed now? What information du you use here, how do you get it, and what do you do with it? Is there anything that makes this particularly difficult? Why is that important? Have I understood you correctly if I were to say

31 31 WHY? Difficult question. Users may start to speculate, but sometimes they know exactly why. Treat the answer with a certain degree of suspicion and triangulate with observational data.

32 32 What to avoid Questions that participants cannot answer to themselves Closed questions Do you like this web site? Do you think this web shop is slow? Leading questions Doesn t this button look like a search button? (leading) What does this button do? (less leading) What do you think about now? (inquiring)

33 33 Avoid asking users to speculate about the future Talk about past events and what they did do. They will most likely keep along the same lines Only ask them to speculate on alternatives in relation to a specific event

34 34 Observations Level of participation Fly on the wall Shadowing Secret agent Apprentice

35 35 Observation focus What order do people do things? What cognitive activites (problem solving, memory, perception, decision making etc.) are involved? What info is used and how is it displayed? What tools are used and what do they contribute with? Are there any workarounds people have developed to compensate for constraints in tools and workplace? What kinds of errors can be made? What other kinds of support could be useful?

36 36 Contextual Inquiry Combining interviews and observation Being an apprentice In context, in place Show and tell Figure out what is important for a participant Focus points, but no strict protocol: semi-structured

37 Analysis

38 38 Affinitity diagramming Categorisation Finding themes Sorting cards or sticky notes Alone, in the design team, or with stakeholders

39 39 1. Read all notes and highlight important passages 2. Every participant reads the highlighted episodes aloud and writes them down on sticky notes. Code the origin 3. Get them up on the wall 4. Group them by identifying similarities or relations. In silence. Keep moving notes until nobody wants to move any more 5. Discuss what binds the groups together: the shared meaning 6. Name the groups using specific notes creating categories. Categories that belong together form themes. There might be sub-categories too. 7. Draw lines between themes, categories and sub-categories to create a hierarchy.

40 Theme 1 Theme 2 Category 1 Category 2 Category 3 Category 4 Idea Idea Idea Idea Idea Idea Idea Idea Idea Idea Idea Idea Idea Idea

41 41 Reviewing the Affinity Diagram What is surprising here? What did we expect to find that we didn t find? What do data tell us that we didn t know before? What in the data make us look at things differently? Why is a certain pattern, or a certain anomaly, appearing?

42 42 Modelling the User Research Think: Concept mapping Feel: UX Analysis with IPA Do: Task Analysis Personas, Scenarios, Storyboards and Customer Journey Maps Setting design objectives

43 Concept Mapping 43

44 hzp:// content/uploads/2008/06/ddo_brand.pdf

45 hzp://cmap.ihmc.us/publica`ons/researchpapers/theorycmaps/theoryunderlyingconceptmaps.htm

46 46 Method: Concept Mapping 1. Focus question. How is thunder formed? 2. Identify the most important concepts in the domain and place them in the parking lot 3. Write them down on sticky notes and move them around to form a first map 4. Search for cross-links between concept in different parts of the map 5. Revise the map, make sure concept-link-concept create propositions 6. Work on the typography and the layout

47 UX Analysis 47

48 How do the users feel and experience the various aspects of the situation? 48

49 norms form and material 1 Aesthe9c Impact character and innovation Ethics power experience of activity 0 habits symbolics -1 Communicaton Technical Build quality constraints possibilities usefullness cooperation and coordination business operations usability Functionality Organization Design proposal: Reviewer: IxDQI Interaction Design Quality Indicator. Mattias Arvola

50 50 Method: Interpretative Phenomenological Analysis Decide what users (3 5) to make a more in-depth analysis of Read and re-read. Carefully. Make first notes: What things are important to the user (relations, processes, places, events, values and principles)? What do they mean for the user and how are they experienced? Why are they important and why are they experienced in this way (how do they talk about it and what is the context)? What underlying abstract concepts can be used to understand the meaning of their statements?

51 51 Develop themes Form Statements about: how things are experienced what those experiences mean for the participant Order them according to their affinity What is similar across participants? What is different? Why?

52 Task Analysis 52

53 53 A task analysis describes Who does what, when, where, how and why How often? How long? How critical? What works well? What doesn t work well?

54 Construct a task set in a flow chart 54

55 Peel the task set into layers 55

56 Remove redundant dependencies 56

57 57 Personas Precise descriptions of hypothetical individuals based on facts about potential users. Portrayed like a real living person with a name and a face to increase empathy Provides a shared understanding on whom to design for A tool for communication in the team and with clients What would Bob the bank clark do? Shaped around goals Primary and secondary personas

58 Matthew Johnson Program Staff Director, USDA 51 years old Married Father of three children Grandfather of one child Has a Ph.D. in Agricultural Economics. Goals and tasks He is focused, goaloriented within a strong leadership role. One of his concerns is maintaining quality across all output of programs. Spends his work time Requesting and reviewing research reports, preparing memos and briefs for agency heads, and supervising staff efforts in food safety and inspection. Environment He is comfortable using a computer and refers to himself as an intermediate Internet user. He is connected via a T1 connection at work and dial-up at home. He uses extensively and uses the web about 1.5 hours during his work day. Quote Can you get me that staff analysis by Tuesday? to- and- tools/methods/personas.html

59

60 60 Method: Personas 1. Identify roles to create personas for 2. Go through the data from users in the selected role 3. Identify variables on which the users differ and set up bipolar scales 4. Place the users on the scales 5. Identify patterns by looking for users that go together on several variables. They will form personas 6. Define goals for the interviewed users whom were close to each other and put them down in the persona

61 61 7. Clarify distinctions between personas and add details 8. Repeat the procedure with the other personas if needed 9. Group and prioritize the personas 10.Develop storytelling and other communication by giving the persona a face and name. Make it come alive.

62 62 Placing on scales and finding patterns Uses only own computer ANVÄNDER ENBART EGEN DATOR Uses always machine translation ANVÄNDER OFTA MASKIN- ÖVERSÄTTNING Reads LÄSER SNABBT quickly Uses only public computers ANVÄNDER ENBART OFFENTLIG DATOR Uses never ANVÄNDER ALDRIG machine MASKIN- ÖVERSÄTTNING translation Reads slow and carefully LÄSER LÅNGSAMT OCH METODISKT

63 63 Triading the users Go trough data from three randomly selected users at the time Find one characteristic that sets one apart from the other two and write down as a bipolar scale Take a new randomly selected triad and do the same Go through ten triads Discuss and come to consensus on which variables to include

64 Scenarios Mr. and Mrs. Macomb are retired schoolteachers who are now in their 70s. Their Social Security checks are an important part of their income. They've just sold their big house and moved to a small apartment. They know that one of the many chores they need to do now is tell the Social Security Administration that they have moved. They don't know where the nearest Social Security office is and it's getting harder for them to do a lot of walking or driving. If it is easy and safe enough, they would like to use the computer to notify the Social Security Administration of their move. However, they are somewhat nervous about doing a task like this by computer. They never used computers in their jobs. However, their son, Steve, gave them a computer last year, set it up for them, and showed them how to use and go to websites. They have never been to the Social Security Administration's website, so they don't know how it is organized. Also, they are reluctant to give out personal information online, so they want to know how safe it is to tell the agency about their new address this way. 64

65 Storyboards 65

66

67

68 Customer Journey Mapping 68 Web Mobile Phone Store user experience

69 hzp:// anatomy- of- an- experience- map

70

71 71 Setting Design Objectives Effect Goals The expected effects of a project Why a design work at all need to be made What it should contribute with to the business and to the users

72 72 Why- why- why Why do people object to traffic congestion?: Because it causes them irritation. Why does it cause irritation?: Because it delays one unexpectedly. Why are unexpected delays irritating?: Because they force one to alter one's plans. Why do road travellers have plans?: Because the car journey is not an end in itself or is it?

73 73 UX Goals How should it be? The UX Analysis gave a list of qualities that should be emphasized or changed in the users experiences. E.g.: Respect for the child Empathy Equality Questioning of authorities Respect for culture Respect for nature Curiosity Breaking the norms

74 74 Objectives Tree Desirable quality 1 Quality criterium 1 Quality criterium 2 Value Desirable quality 2 Quality criterium 3 Quality criterium 4 Desirable quality 3 Quality criterium 5 Quality criterium 6

75 75 Business Impact Mapping What are the desired effects and values that the design should contribute to? Who are the stakeholders who can create this value and these effects? What are their goals? How can the design contribute to them?

76 Business goals 76 Target groups Use goals Scenarios and func9ons Fritt baserat på en beskrivning av Antrop.

77 77 Tydliga svar Vill veta vad som finns att söka för just henne, i hennes situation Vill bli guidad i hur hon ska göra när hon ska söka bidrag Göra uppföljningar i det projekt han leder Ta del av nyheter i regionen Få svar på projektspecifika frågor från finansiärer Fia, 37 år VD Mekoväst Calle, 56 år Projektledare i regional planering Webbplatsen Effektmål 1 (E1) På webbplatsen framgår tydligt vilken roll myndigheten har. Effektmål 2 (E2) På webbplatsen är det tydligt vilka stöd som finns för företagare och folk på regionala myndigheter. Effektmål 3 (E3) Det ska vara enkelt att göra en ansökan och delta i program hos myndigheten. Effektmål 4 (E4) Genom mål E1-E3 bidrar webbplatsen till att arbetsbelastningen på handläggare minskar. Mia, 43 år Strateg för företagarfrågor E1, E2, E3 (E1), E2, (E3) Martin, 43 år Chef på utvecklingskontor E2, (E3) (E1), E2, (E3) Få en långsiktig överblick på hur myndighetens stödverksamhet ser ut Få svar på områdesspecifika frågor Nätverka med andra som är aktiva inom samma område Ta del av fakta och statistik Bevaka vad som händer i regionen Vara påläst om aktuella utlysningar och ansökningsomgångar

78 78 Building an impact map Adzic (2012) As a <who?> I want <what?> So that I can <how?> The <what?> can help me do that.

79 79 Design is a learning process Revise objectives Specify objectives Make them concrete as you learn more

80 @mattiasarvola