Qualitative User Research Methods Mattias Arvola
2 The process has to begin with some kind of immersion in context and exposure to people who are perhaps experiencing that aspect of life right now. It starts by finding out where we are gonna go, who we are going to speak to, what is their world and really stepping into their shoes. For us to start a design project without that is a bit like cutting of our arms and legs and probably our head as well, we don't know where to start in that case. Anonymous User Experience Designer
3
4 Visit your users. Speak with them. Write down what you have seen and heard.
5 Cost for Change Requests Time
6 Cost for Change Requests Impact of UX Time
7 Value is created in use by your users. Knowing them, their goals and activities is vital for success.
8 Aims of User Research Gain insights on: What we should to design/build/develop Why? How to do it?
9 Research Questions Qualitative What? How? Why? Quantitative How much? How long? To what degree? What affect does X have on Y?
Qualitative Research 10
11 Interpretative research Research is interpretative when the understanding of human sense making is gained through the study of social constructions (e.g. language, shared meanings, consciousness, artefacts etc.) as a situation emerges.
12 The Hermeneutical Circle Whole Parts
13 Characteristics of Qualitative Research No sharp boundaries of study objects Careful contextualization of the instance Transferability rather than generalizability Cooperation between researcher and participants Purposeful sampling Alternative interpretations Suspicion of interpretations Going beyond what is said and done to what is intended and motivated
14 Validation Strategies Prolonged engagement and observation Building trust Learning the culture Checking misinformation and distortions Finding the focus Triangulation Researcher Theory Method Peer review, debriefing and external audits Clarifying what subjective things that have likely shaped the the interpretation past experiences, biases prejudices orientations Member checking Rich and thick description Revise working hypothesis Negative cases with disconfirming evidence
Preparations 15
16 Initial Framing What is the project about? Finding the target group Not the most representative, but the most important users Who s needs are the most important for us to succeed? Who might have something interesting to say? Who are the ones most affected by the project? Prioritise
17 Sensitizing Persona hypothesis Imagine mainstream and extreme users What do they do? In what situations? How do they do it? Why do they do it? What characterizes them in terms of motivations, knowledge and skills. What do they lova an hate?
18 How many should we meet? Breadth and depth vs. cost Three participants from three places/contexts As different as possible but within the market segment Recruiting takes time
19 Ethics and Informed Consent Knowing that you are observed Understanding what it means Potential risks Gains Anonymity or not Permission to record Provide the participants with the results Introduction in writing
Gathering Data 20
21 Setting a Focus Getting information and inspiration for design that gives leverage Start with the brief and discuss it with clients Find things to handle in the project
22 The Persona and Scenario Sheet A checklist with themes to cover The scenario part: What people do The persona part: How people are Who does what, where, when, how, and why (5WH) Download from: http://www.ida.liu.se/~matar/tools.en.shtml
Foundation for personas Goals and driving forces Skills and knowledge Experiences Quote Portrait Family and contacts Likes Name: Description: Dislikes Age: Role: Sex: Habits Income: Hometown: Background Project: Date: Researcher: Persona and scenario sheet. Mattias Arvola 2014.
An ordinary day for the persona An ordinary day starts with Project: Date: Researcher: Persona and scenario sheet. Mattias Arvola 2014.
Foundation for scenarios OBSERVATIONS AND FACTS WHAT is done? INTERPRETATIONS Weaknesses to address WHY is it done? (participants' motivation and researchers' interpretation) Strengths to emphasize HOW is it done? Constraints to deal with WHO does it and who are involved? WHEN and WHERE is it done? Project: Date: Researcher: Persona and scenario sheet. Mattias Arvola 2014.
26 What to look for Observations and facts Opportunities for change Things to preserve and highlight Constraints What characterizes a good/bad situation? Use a protocol for semi-structured interviews and observations (the persona and scenario sheet?) Phenomena and patterns
27 Field notes Separate observations and interpretations Use pseudonyms Quotes Sketching with comments and detail studies Note the background, steps and contexts of activities Write the notes up as soon as possible
28 Recordings Video, audio or neither Quotes Intrusiveness Transcription and levels of it Clips of episodes No replacement for field notes
29 Interviews Open questions that make participants tell stories and explain Tell me about the first time you Tell me about a critical situation when matters were brought to a head Could you describe when you could not Tell me about the last time you Tell me about when it went really well or really bad
Keep them talking 30 How do you mean when you say What are you doing now? How do you think about this situation? What are you taking notice of in this situation? Are there any rules of thumb for how to proceed now? What information du you use here, how do you get it, and what do you do with it? Is there anything that makes this particularly difficult? Why is that important? Have I understood you correctly if I were to say
31 WHY? Difficult question. Users may start to speculate, but sometimes they know exactly why. Treat the answer with a certain degree of suspicion and triangulate with observational data.
32 What to avoid Questions that participants cannot answer to themselves Closed questions Do you like this web site? Do you think this web shop is slow? Leading questions Doesn t this button look like a search button? (leading) What does this button do? (less leading) What do you think about now? (inquiring)
33 Avoid asking users to speculate about the future Talk about past events and what they did do. They will most likely keep along the same lines Only ask them to speculate on alternatives in relation to a specific event
34 Observations Level of participation Fly on the wall Shadowing Secret agent Apprentice
35 Observation focus What order do people do things? What cognitive activites (problem solving, memory, perception, decision making etc.) are involved? What info is used and how is it displayed? What tools are used and what do they contribute with? Are there any workarounds people have developed to compensate for constraints in tools and workplace? What kinds of errors can be made? What other kinds of support could be useful?
36 Contextual Inquiry Combining interviews and observation Being an apprentice In context, in place Show and tell Figure out what is important for a participant Focus points, but no strict protocol: semi-structured
Analysis
38 Affinitity diagramming Categorisation Finding themes Sorting cards or sticky notes Alone, in the design team, or with stakeholders
39 1. Read all notes and highlight important passages 2. Every participant reads the highlighted episodes aloud and writes them down on sticky notes. Code the origin 3. Get them up on the wall 4. Group them by identifying similarities or relations. In silence. Keep moving notes until nobody wants to move any more 5. Discuss what binds the groups together: the shared meaning 6. Name the groups using specific notes creating categories. Categories that belong together form themes. There might be sub-categories too. 7. Draw lines between themes, categories and sub-categories to create a hierarchy.
Theme 1 Theme 2 Category 1 Category 2 Category 3 Category 4 Idea Idea Idea Idea Idea Idea Idea Idea Idea Idea Idea Idea Idea Idea
41 Reviewing the Affinity Diagram What is surprising here? What did we expect to find that we didn t find? What do data tell us that we didn t know before? What in the data make us look at things differently? Why is a certain pattern, or a certain anomaly, appearing?
42 Modelling the User Research Think: Concept mapping Feel: UX Analysis with IPA Do: Task Analysis Personas, Scenarios, Storyboards and Customer Journey Maps Setting design objectives
Concept Mapping 43
hzp://www.dubberly.com/wp- content/uploads/2008/06/ddo_brand.pdf
hzp://cmap.ihmc.us/publica`ons/researchpapers/theorycmaps/theoryunderlyingconceptmaps.htm
46 Method: Concept Mapping 1. Focus question. How is thunder formed? 2. Identify the 15 25 most important concepts in the domain and place them in the parking lot 3. Write them down on sticky notes and move them around to form a first map 4. Search for cross-links between concept in different parts of the map 5. Revise the map, make sure concept-link-concept create propositions 6. Work on the typography and the layout
UX Analysis 47
How do the users feel and experience the various aspects of the situation? 48
norms form and material 1 Aesthe9c Impact character and innovation Ethics power experience of activity 0 habits symbolics -1 Communicaton Technical Build quality constraints possibilities usefullness cooperation and coordination business operations usability Functionality Organization Design proposal: Reviewer: IxDQI Interaction Design Quality Indicator. Mattias Arvola. 2011-04-15
50 Method: Interpretative Phenomenological Analysis Decide what users (3 5) to make a more in-depth analysis of Read and re-read. Carefully. Make first notes: What things are important to the user (relations, processes, places, events, values and principles)? What do they mean for the user and how are they experienced? Why are they important and why are they experienced in this way (how do they talk about it and what is the context)? What underlying abstract concepts can be used to understand the meaning of their statements?
51 Develop themes Form Statements about: how things are experienced what those experiences mean for the participant Order them according to their affinity What is similar across participants? What is different? Why?
Task Analysis 52
53 A task analysis describes Who does what, when, where, how and why How often? How long? How critical? What works well? What doesn t work well?
Construct a task set in a flow chart 54
Peel the task set into layers 55
Remove redundant dependencies 56
57 Personas Precise descriptions of hypothetical individuals based on facts about potential users. Portrayed like a real living person with a name and a face to increase empathy Provides a shared understanding on whom to design for A tool for communication in the team and with clients What would Bob the bank clark do? Shaped around goals Primary and secondary personas
Matthew Johnson Program Staff Director, USDA 51 years old Married Father of three children Grandfather of one child Has a Ph.D. in Agricultural Economics. Goals and tasks He is focused, goaloriented within a strong leadership role. One of his concerns is maintaining quality across all output of programs. Spends his work time Requesting and reviewing research reports, preparing memos and briefs for agency heads, and supervising staff efforts in food safety and inspection. Environment He is comfortable using a computer and refers to himself as an intermediate Internet user. He is connected via a T1 connection at work and dial-up at home. He uses email extensively and uses the web about 1.5 hours during his work day. Quote Can you get me that staff analysis by Tuesday? http://www.usability.gov/how- to- and- tools/methods/personas.html
60 Method: Personas 1. Identify roles to create personas for 2. Go through the data from users in the selected role 3. Identify variables on which the users differ and set up bipolar scales 4. Place the users on the scales 5. Identify patterns by looking for users that go together on several variables. They will form personas 6. Define goals for the interviewed users whom were close to each other and put them down in the persona
61 7. Clarify distinctions between personas and add details 8. Repeat the procedure with the other personas if needed 9. Group and prioritize the personas 10.Develop storytelling and other communication by giving the persona a face and name. Make it come alive.
62 Placing on scales and finding patterns Uses only own computer ANVÄNDER ENBART EGEN DATOR Uses always machine translation ANVÄNDER OFTA MASKIN- ÖVERSÄTTNING Reads LÄSER SNABBT quickly 2 4 1 3 5 3 5 1 2 4 5 3 1 4 2 Uses only public computers ANVÄNDER ENBART OFFENTLIG DATOR Uses never ANVÄNDER ALDRIG machine MASKIN- ÖVERSÄTTNING translation Reads slow and carefully LÄSER LÅNGSAMT OCH METODISKT
63 Triading the users Go trough data from three randomly selected users at the time Find one characteristic that sets one apart from the other two and write down as a bipolar scale Take a new randomly selected triad and do the same Go through ten triads Discuss and come to consensus on which variables to include
Scenarios Mr. and Mrs. Macomb are retired schoolteachers who are now in their 70s. Their Social Security checks are an important part of their income. They've just sold their big house and moved to a small apartment. They know that one of the many chores they need to do now is tell the Social Security Administration that they have moved. They don't know where the nearest Social Security office is and it's getting harder for them to do a lot of walking or driving. If it is easy and safe enough, they would like to use the computer to notify the Social Security Administration of their move. However, they are somewhat nervous about doing a task like this by computer. They never used computers in their jobs. However, their son, Steve, gave them a computer last year, set it up for them, and showed them how to use email and go to websites. They have never been to the Social Security Administration's website, so they don't know how it is organized. Also, they are reluctant to give out personal information online, so they want to know how safe it is to tell the agency about their new address this way. http://www.usability.gov/how-toand-tools/methods/scenarios.html 64
Storyboards 65
Customer Journey Mapping 68 Web Mobile Phone Store user experience
hzp://www.adap`vepath.com/ideas/the- anatomy- of- an- experience- map
http://adaptivepath.org/uploads/images/raileurope_adaptivepath_touchpoints_v1.png
71 Setting Design Objectives Effect Goals The expected effects of a project Why a design work at all need to be made What it should contribute with to the business and to the users
72 Why- why- why Why do people object to traffic congestion?: Because it causes them irritation. Why does it cause irritation?: Because it delays one unexpectedly. Why are unexpected delays irritating?: Because they force one to alter one's plans. Why do road travellers have plans?: Because the car journey is not an end in itself or is it? http://deseng.ryerson.ca/dokuwiki/design:why_why_why
73 UX Goals How should it be? The UX Analysis gave a list of qualities that should be emphasized or changed in the users experiences. E.g.: Respect for the child Empathy Equality Questioning of authorities Respect for culture Respect for nature Curiosity Breaking the norms
74 Objectives Tree Desirable quality 1 Quality criterium 1 Quality criterium 2 Value Desirable quality 2 Quality criterium 3 Quality criterium 4 Desirable quality 3 Quality criterium 5 Quality criterium 6
75 Business Impact Mapping What are the desired effects and values that the design should contribute to? Who are the stakeholders who can create this value and these effects? What are their goals? How can the design contribute to them?
Business goals 76 Target groups Use goals Scenarios and func9ons Fritt baserat på en beskrivning av Antrop.
77 Tydliga svar Vill veta vad som finns att söka för just henne, i hennes situation Vill bli guidad i hur hon ska göra när hon ska söka bidrag Göra uppföljningar i det projekt han leder Ta del av nyheter i regionen Få svar på projektspecifika frågor från finansiärer Fia, 37 år VD Mekoväst Calle, 56 år Projektledare i regional planering Webbplatsen Effektmål 1 (E1) På webbplatsen framgår tydligt vilken roll myndigheten har. Effektmål 2 (E2) På webbplatsen är det tydligt vilka stöd som finns för företagare och folk på regionala myndigheter. Effektmål 3 (E3) Det ska vara enkelt att göra en ansökan och delta i program hos myndigheten. Effektmål 4 (E4) Genom mål E1-E3 bidrar webbplatsen till att arbetsbelastningen på handläggare minskar. Mia, 43 år Strateg för företagarfrågor E1, E2, E3 (E1), E2, (E3) Martin, 43 år Chef på utvecklingskontor E2, (E3) (E1), E2, (E3) Få en långsiktig överblick på hur myndighetens stödverksamhet ser ut Få svar på områdesspecifika frågor Nätverka med andra som är aktiva inom samma område Ta del av fakta och statistik Bevaka vad som händer i regionen Vara påläst om aktuella utlysningar och ansökningsomgångar
78 Building an impact map Adzic (2012) As a <who?> I want <what?> So that I can <how?> The <what?> can help me do that.
79 Design is a learning process Revise objectives Specify objectives Make them concrete as you learn more
@mattiasarvola www.liu.se