Jarrett Interaction Design

Gina Villavicencio, Director UX Design and James Nixon, General Manager, Digital Globalization, Marriott International

– share an accomplishment or two:
– ratings and reviews product; 2/3 of millenials use ratings and reviews before choosing a hotel; search visibility; Marriott not well known in some international markets; want to see reviews from people like them; ratings and reviews can hurt conversions, so goal was “do no harm”; increased conversion; feature designed by US team, did competitive analysis in international markets; had to look at customers who weren’t familiar with Marriott; tested in Germany, France, Italy, and China; tested our review version vs. local; source doesn’t matter as much as it coming from a person like the user
– integration with WeChat: China, 549 million monthly users, 1 billion registered users; texting + photo/audio/video + payments + other; English version only has two features, not the full ecosystem available in China; not a lot of creative opportunity with integration, working from template; first time the region is leading the project and HQ is learning from them; WeChat is what Facebook will be in a few years, and we are learning through this; in China, mobile is more important than web
– where does Globalization sit in org: starts at CEO; growth is coming from international markets; decentralized structure with presidents and support staff in different regions; mandated HQ to support those regions; Digital Globalization is horizontal across the digital experiences but also vertical and builds experiences and products; centralized core technology, but decentralized for specific purposes in the regions; growth is in regions, but scale is focused at HQ
– UX team is structured similarly; UX leads for each product line, standards team that crosses all, and global UX team that works across and vertically as well; principle, everyone is responsible for global, so hope that someday no need for a specific global UX team; started with research to infuse global information, but it’s now more structured – still research based, but has packaged training, KPIs, and rewards
– localized prototypes: trying things in the regions and decide if UX should be decentralized, including specific technologies and changes in specific regions; e.g. localized content, regions have more control over the content of the site (text and imagery); need a governance model that enables regions but maintains scale and quality
– how do you start and then build momentum: need executive support; fear of loss of control, especially around brand; intelligent experimentation then figure out how to formalize and scale, then monitor
– how do you get exec support: James’s group has to align the different regions and groups to get the funding for these initiatives; budgets supposed to take 3 months but takes longer; lots of education; measure impact in 3 buckets: core traditional metrics (“win the booking, win through the stay”), adoption metrics (largely education right now), strategic bets metrics (big local initiatives); 80/20 rule definitely applies in globalization, 80% decisions can be global, 20% need to be local
– global UX initiatives: influence product decisions (including domestic), impact the entire platform; localized prototypes are key; e.g. enhancements to search forms for local needs; work with standards teams to ensure quality but enable exceptions to exceptions; it’s not a standard if it’s not validated globally; e.g. favorite icon = heart, or does it need a label; e.g. Brazil payment is payment system “layaway on steroids”, need to support multiple payments for hotel stays; took about a year to convince the organization to support it; 50% of James’s job is education
– significant and growing part of revenues is coming from outside the US
– what obstacles to overcome: without local support teams for product, do an experiment with temporary or agency help, and show the multiplier; need initial funding, get creative; get a regional account but manage it domestically
– what do you do if the region pushes back: had to prove ourselves, for example in China; again the local prototyping, research in market, build a prototype, test that prototype; built credibility

Steve Turbek, SVP UX Design, Fidelity Investments

– 3 1/2 years with Fidelity
– 60,000 employees
– Customer Experience is not just designing screens, but a motivating program across the whole organization; call center; in-person branch experience
– naturally led to taking user centered design techniques to thinks like paper forms
– management was way ahead of wanting to drive customer satisfaction; UX middle-management has to translate business desires to execution; in the past he focused too much on selling UX and not ‘how can I help you’
– look for people who need something you can help with, then help them, rather than trying to convince them they need what you have
– his role is largely connecting individuals across different departments and functions; take the tools we use and apply them in new ways; e.g. user research to help business folks solve business problems, not just evaluating interfaces
– started with many interviews and such to discover the problems
– where is the team within the org: org design is very important; 300+ people, centralized UX design group in very decentralized business; be strong where we need to be strong but flexible in other ways; there was no perfect place for the team, but they chose a “good place” within one of the digital product teams; evaluate for the president each year whether the UX team is serving the needs of the other groups well
– design has two roles: 1) the technical product design and project implementation, and 2) advanced visualization of concepts; need to separate the two; can’t jam advanced concept visualization into an ongoing project
– moving UX into a product group instead of a central service is key; getting the product group to invest in UX work before there is a “project” in development
– how does your oversight role work: key role is to keep the different silos communicating; forcing function; balance the communication and delivering results
– how do you attract and motivate designers: are you solving a problem that people are really intersted in; does the organization value and invest in design, that it “gets” design; Fidelity takes care of people, and invests in the space for creative work; my office reflects that I spend my whole day talking to people; round table, conference phone, white board, book shelves, remote control collection on wall (as inspiration that there are lots of hard to use controls, just as there are hard to use financial product)
– transitioning to mobile is a big deal; created a ‘device bar’ in the middle of the space to show that what they work on has to work on all those devices; moved them from a lock box to an open display; “culture is a story you tell yourself and everyone repeats over and over”; cultural focal point; teaching tool when other groups come in for a tour
– Fidelity UX group has been around for decades, did you make changes to the roles and titles and what they imply: every designer’s not a vice president :); roles and titles are different things; treating people fairly with titles and such is important in a big organization; understanding roles means that the title doesn’t restrict what you do; e.g. act like a creative director and guide the code changes, don’t create mockups to just move a few things around; his main role is to keep the culture going well and develop the young designers; not all folks want a management role; need a contributor path to advance in their career; worked with HR to understand that the more senior a contributor is, the more different roles they can fill, so you can pay an individual more that fills more roles yet reduce the overall cost
– you have to pay designers well, as well as create a great culture; design is a real career; need to respect the people you work with and make sure the organizational culture supports that; does the organization stand for something you can believe in; e.g. individual can bang a gong and recognize good work by another, makes it a public and positive
– how did you roll out the space design and make it work: the space is not just for design; design hosts people coming together at certain points in a project; people locate there for an entire agile project; innovation space, not design space
– how do you deal with partnerships and acquisitions: work with many startup companies on specific things, but difficult to scale; what works for one customer segment may not work at all for others; Fidelity has diverse consumers, from retired folks to kids; design role is to drive the requirements and make that knowledge of the differences and commonalities available
– use a lot of internal design groups and many agencies; have an explicit process for evaluating design firms and saying what they are good at; when someone is looking for an external firm, they have a list of recommended ones that are vetted and agreements in place already
– what do you look for in an agency: folks go out and talk to different agencies; different agencies have different specialties; biggest thing is ensuring the way you work together is good; seed people across teams; share pattern libraries and tools
– customer-centricity taken too far, like at Amazon: comes right from the top of the organization, treating people well; people come and want to stay; previous experience included a financial service company that didn’t, and it was dynamic, but probably not a long-term success culture
– individuals with multiple roles on a project: ideal agile is no individual is on multiple projects, but not realistic; they are senior enough to know how to do things more effectively and productively; tension between coming up with new ideas and testing with users; creative frission; try to keep user researchers seperate so they don’t test something that they themselves designed
– how do you mentor and coach more junior designers: constantly hiring as a large company; two part strategy, 1) always look for the best folks in the industry, and 2) hire really junior people via internships and straight out of college; junior folks don’t have the tools and skills to succeed, so need to invest a lot in training and mentoring; every new one gets partnered with a more senior designer; investment pays of quickly; as you become more senior, you are expected to be a mentor; may teach a class a few times a year

Bill Scott, VP of Next Gen Commerce, PayPal

– 4 years at PayPal
– key accomplishement: reinvented “check-out” process – slight variance in conversion is a lot of $$$
– war room: 2-3 designers, 3 engineers, 3 product folks (balsamiq mockups from new CEO); prototype in a day; usabilty testing within a week; whiteboard to prototype, no specs and such
– forensic investigation of existing design process; create things, throw over the wall
– need executive support + ground-swell from the folks on the ground; have to have some principles you believe in; organizations are set up to maintain the status quo; shared understanding between all functions of how you get product out the door; continuous customer feedback; adaptive to current situation, not rigid playbook; “not sure what I’m going to do, but this is what I believe”; wisdom in crowds; many people want to do the right thing; dogged persistence + humility and improv
– PayPal experince in 2011 felt like 1999
– adjust the playbook as things change
– principles that drive the playbook: get engineering, product, and design working together, other functions like legal as well (legal became a design team); always make sure you ask what problem are you trying to solve; get customers involved (field research, user testing, etc)
– put a smart team together, soak them in the users’ problems, you’ll have a good product
– when from 500 people on design team to 10 reallly strong cross functional people
– how do you start working with a new culture when you start the job: people are a common element of all orgs (we’re all messed up in different ways; each has an agenda/self-interest); don’t sacrifice principles or the product; sometimes make a key decision successful and don’t be in the forefront; don’t worry about who gets the credit; identify those few people in the organization who won’t change, but have to route around (and hope for them to leave)
– how do get executives to support you: speaking the language of the customer is speaking is speaking the language of the business; UI layer is the experimentation layer; “tweetable moments” sound bites; moving from culture of delivery to culture of learning; where multiple hats as necessary, come from each peson’s angle; connect at base need level
– organizational structure influence: who you end up being in exec staff each week affects the whole organization; real work happens across organizational boundaries; identify where the connections aren’t happening; did 100 interviews in the first 45 days, across all parts of the org at all levels, for a short a time as 30 minutes; organizational map; lots of weird overlaps in a large organization; learn a hell of a lot; apply design research methods to changing the organization; then diagnose and intervene
– what did you discover about design in those interviews: design and front-end engineering were not well-respected; developers (engineers) vs “web dev” (wrote templates); changed “web dev” into “user interface engineers”, bring in good people and solve hard problems; on design side, many different leaders with different attitudes; design done at the end to make things pretty; new leadership came in and helped make the change
– how do you get alignment among various UX teams that don’t report into the same places in org; PayPal was trying to create a pattenr library (really becoming Pattern Police); if the core design team isn’t solving real problems for those other groups and helping make them better, they will fail; start thinking of those other teams as customers, understand how they work, and improve their work; make them more successful and they will embrace you
– does it get political around design at PayPal: 18-19 anti-patterns on Lean UX (http://www.slideshare.net/billwscott/lean-ux-antipatterns); different actors play a big role; “genius designer” (usually from Apple :)) – won’t hear usability feedback; need a dose of customer/user reality
– can you do something through hiring: need design leaders who’ve worked with engineers and product; some from agencies and such retreat back into their studio
– how did you get the company to change from long-planning process to lean ux: if you don’t have high-level support giving you carte blanche (like he did at PayPal “make us love usability”), you need to create a sandbox to try something and show the difference, hopefully something important
– how important is evangelism: coming to PayPal, lots of front-end engineers were leaving; had to change the story; lots of presentations but CEO wanted the message of the change to be heard publicly; got in trouble with PR a few times
– transformed in 6 months with 7 people what 100 people couldn’t do in 4 years (moving to new technology stack)
– any resistance to war room, and has it propagated to other projects: president got them the best conference room and offered the board room; results are best when people are co-located and right next to each other (less than nuisance distance)
– how do you deal with the disruption to existing flows when an outside firm is brought in from the outside: intervene and ensure that a) real problems are being solved and are the focus, and b) collaboration across the groups must be maintained, and c) the deliverables from the outside group need to be understood, consumed, and used
– stripe was founded by folks who tried to use the paypal API and said they could do better; PayPal didn’t have enough fear of disruption; as long as we engage and enable every type of transaction, we win; can’t obsess over the fear; payments is a hard problem
– most exciting thing coming up that you can talk about: next gen commerce – enable people who don’t have good livelihood now to get access to money through loans and such; working with the unbanked and underbanked; equal access is exciting

Karen McGrane
– wanted to make an event for people who are already making it happen as leaders in their orgs
– 6 interviews – executive sessions, huge global corporations where design initiatives are big and working; 4 interviews on more focused topics
– gaining executive support: frustrating: orgs want design but aren’t will to make changes to pull it off; how do you get budget; how do you convince; is there a crisis; what space do you get to play (and fail) in
– reinvent corporate structures: small local group -> central team with budget -> pervasive design thinking across organization; where does the UX function and leadership live
– restructuring incentives and rewards: hard incentives: how are execs rewarded; how is success recognized and rewarded; how to not focus just on shipping but quality of its product; soft incentives: how to reach the hearts of individual designers; how do you know its working; how do you motivate, even across stronger vs weaker team members
– shift to continuous deployment: it’s a very deep change; how do you make it work for more than developers; can you tweak your way to design success; big companies and the government
– taking advantage of fear: what if you are the next healthcare.gov; what about a new competitor or market disruption; can be a catalyst for change but hard to admit
– governments’ design lessons: seem to be the slowest movers, but have done some amazing things; US Digital Service and Los Angeles County; fascinated by people who don’t come from a design background but realize there are techniques and tools out their to make their work and business better
– inventing the ‘yes’ lawyer: legal is a blocker to many ideas; how do you get legal on your side; struggle with contracting especially in highly regulated industries; contracts that allow iterative and creative work rather than laundry lists of requirements; how to not be in conflict between UX and legal; incorporate the actual meaning and spirit of the law into design; agile/iterative methods
– designing a global UX: many companies have plans but have not achieved; how do you operate a global business right; how do customers in different places make decisions; executive buy-in; where does globalization live in org; how can it be infused
– the role of outsiders: is there still a role for design agencies; eg. Capital One acquired Adaptive Path; how to think about internal vs external resources and teams; reasons to use outside agencies beyond just extending capacity

“The Physical Web”

• Apple, Fro Design, now Google
• contrast experience of Amazon Whispersync (zero experience, it just works) vs Jabra headphones (waking his wife up in other room)
• mobile moving too fast to standardize yet, eg pull to refresh is great but won't be here forever; even steering wheels haven't been standardized, need for it will go away, and history is quite varied; dialectic between users and technology
• shape of innovation: familiarity -> maturity -> revolution; eg DOS -> Lotus 123 -> GUI; lesson 1 – we'll always borrow from the past, lesson 2 – maturity is an intellectual gravity well that's hard to escape
• limiting factor isn't technology but our own psychology; everyone wants innovation but not risk; not afraid of future but attached to the past
• Internet of Things changes everything; not a lot of good thinking evidenced in the media today; smart devices – Nest and Quirky Egg Minder, individual functional devices; home automation – everything connected and networked; we need to think about the implications and consequences with all this stuff in combination
• IoT can't be a set of if/then rules because humans are goofy and do unexpected things
• Moravec's Pardox: HardEasy (think it's hard but turns out easy like chess) vs EasyHard (think it's easy turns out hard like translation) – home automation is an EasyHard problem; systems that expect us to be human rather than forgetting it
• smart devices: today each device has its own app; can't sustain that, they don't scale to millions of smart devices
• just-in-time interaction: use it then lose it, don't need to hold on to it or remember it after your done
• smartness layers: coodination (whole environment collaborates), control (one device), discovery (things project tiny bits of data); lose apps and we can think small; web needs a discovery service, smart devices project a URL and phone can make them available; “proximity DNS”; URLs are flexible, lightweight, extensible, and standardized
• “I'm more of a terraforming guy than a VC”; ong term, big change thinker; only 2 kinds of ideas – truck ideas and road ideas; no one wants to build roads right now, just trucks and toll roads; eg Malcom ??? invented and patented cargo container; reduced shipping costs by 26x; gave his patents away to ISO; was even more successful
• Apple's success has blinded us; we need to discover, invent, and move on to new things; need a physical web

Pervasive Information Through Constant Personal Projection by Christian Winkler

• AMP-D: interactive personal ambient display that projects on floor near device; constant personal projection; course augmented reality; interaction on floor, hand, mobile
• mobile devices disconnect from immediate environment; this helps reconnect
• the world as a display: content – static, environment, dynamic, urgent; where – the ground; how – boxes and spheres; when – static, relative to fixed location, with user, with timeouts; interaction – body movement, selection with hand, preview and binary decisions with hand gestures, transfer to/from phone, deselect/remove/snooze; privacy – no projection of private info on floor, only in hand
• implementation: DLP projector with servo focus, depth camera, inertia sensor, hand/finger tracking; continuous interaction space, continuous information space

Bigger Is Not Always Better: Display Size, Performance, and Task Load During Peephole Map Navigation by Roman Radle

• dynamic peephole navigation: display is window to larger information space; how small can a peephole be without overburdening for navigation; tablet size seems to be the sweet spot
• navigation behavior: learning – scan the space, navigation – memory and landmarks for direct access
• experiment: simulated peephole size on a large display with 3D pointer; navigate to 4 target pins as quickly and accurately as possible with 4 distractor pins; vary peephole size from projector to mobile projector to tablet to phone
• results: long learning phase time lengths dropped to stable navigation phase; larger peepholes facilitate learning by reducing path length to view information space and better performance; no significant difference in navigation phase performance

Mechanical Force Redistribution: Enabling Seamless, Large-Format, High-Accuracy Surface Interaction by Alex Grau

• MFR: high density force interaction with low density sensors; arbitrarily large sensor matts at relatively low costs; can be used with many sensor types; scan and interpolate between the forcels (force pixels); resolution dependent on force sensor and space between them
• cool demo of 121ppi hand sensor; multi-touch and hires position and pressure tracking
• uses: automotive interiors; display walls; industrial; yoga mat sized for consumers, developers, and researchers – kickstarter later this year, hope to sell for $250 each

Effects of Display Size and Navigation Type on a Classification Task by Can Liu

• displays getting larger and higher resolution; larger displays promote physical navigation but problematic for some uses such as desktop tasks; previous research hasn't looked at data manipulation tasks
• is a wall display than a desktop for classification tasks?
• experiment: abstract classification task; does a wall outperform a desktop in high information density and task difficulty? 12 participants
• results: desktop worked best on low info density; wall worked much better for high info density
• why? different number of pick and drop actions? no difference; virtual zoom distortion? no difference; physical move distances? no difference at high density; reach range and trajectories? desktop condenses reach range requiring more pan and zoom, with more restrictions on trajectory, even if using overview or fisheye techniques

Model of Visual Search and Selection Time in Linear Menus by Gilles Bailly

• model to understand human performance for target acquisition in realistic menus
• novice: scan, skip around; intermediate: directed search with some error; expert: directed search with less error or point directly
• gaze distribution = f(menu organization, menu size, position of target, absent items, expertise); last item effect – last item is slightly faster to select
• data collection: 40,000 selections for time, cursor position, and gaze position; cursor follows gaze
• model handles previous findings about menu usage; accurate describes behavior, not a simple model but has 3×8 parameters for a complex task

Towards Accurate and Practical Predictive Models of Active-Vison-Based Visual Search by David Kieras

• color is a better cue than size or shape but all contribute; want to build a model to predict human performance; built an EPIC model for this task; very good fit to empirical data; EPIC models are complex and hard to develop; want to develop a GOMS model that then can generate a GLEAN GOMS
• color can be distinguished in a much wider angle than size and shape; focus model on color alone and comes close enough for many situations; useful for model-based evaluation

Understanding Multitasking Through Parallelized Strategy Exploration and Individualized Cognitive Modeling by Yunfeng Zhang

• in many tasks, multi-tasking is inevitable; computational cognitive models allow study
• experiment: multimodal duel task; classification + tracking; sound on or off; peripheral (other display) visible or not
• result: sound helps when peripheral not visible for both tasks; combine even better
• EPIC model: explore 72 different microstrategies for task switching, with 12 settings, so 864 models; used parallel computation to speed up the simulations, shortening from 14 hours to 20 minutes
• basic model follows human data closely; can also compare different strategies; human data averages tracks best strategies closely, but individual performance varies widely
• individualized models fit data well and could find best strategies by comparing best human performers; average performance leads to a match to bottom performing human

How Does Knowing What You Are Looking For Change Visual Search Behavior by Duncan Brumby

• 2 types of search: semantic vs known-item search; known-item is faster; why are semantic searches slower?
• accessing facts in our head takes time; is it reflected in eye movements? no, except when tightly packed
• instead, it relates to the distance between eye jumps; semantic goes item by item, known-item jumps around

Automated Nonlinear Regression Modeling for HCI by Antti Oulasvirta

• nonlinear regression models: expressive and white-box, like pointing, learning, foraging; hard to acquire these models
• exploration is inefficient and laborious, so automate it; using optimization techniques from symbolic programming
• experiment: 11 existing models in literature using same data; improved 7 of 11 models and nearly the same for 4 others; complex data sets come up with complex models; constrain settings; also works with multiple data sets

An Ethnographic Study of South African Mobile Users by Susan Dray

• [10 minutes of technical problems; harumph]
• consulting with undisclosed client
• study from 2008 to inspire ideas on entering African market; interested in mobile devices; broad scope, with tentative ideas in safety and finance; 3 months from first encounter to final report
• Khayelitsha Township
• assumptions: rural people are unbanked FALSE; travel long distance on foot TRUE; need to send money by car/bus FALSE
• 11 families in informal area shacks, formal areas, ADP housing; also 3 families in rural area receiving money; all had basic or feature phones
• challenges: feasibility – (approvals took a long tome plus other logistics), access to participants (recruiting), localization and translation (Xhosa); logistics; safety; trade-offs;
• results: identify new product areas; body of knowledge on urban and rural; develop empathy for people at the bottom of the period

Adopting Users' Designs to Improve a Mobile App by Kate Sangwon Lee

• Haver Corp: make many apps include Line and Naver App
• many changes to apps over time; for small changes user research is often skipped
• developed quick and participatory method; 44 users in 3 days including prototyping; cafe study + participatory design
• method: interview (10 min) -> participatory design (15 min) -> concept evaluation (5 min)
• Challenges: approaching stranger in cafe (interview 1 or 2 at a time, use cafe cards for pay, keep it short); prototyping (printed background of UI, large enough to record descriptions, colored pencils)
• results: 1 dominant pattern (frequently accessed functions) and 2 minor patterns (practical info like weather and horoscopes); prototype and test 3 different prototypes
• strengths: cheap and fast; easily identify subtle needs; visual outputs easy to understand and share; easy to conduct; mobile; multiple domains – mobile, small PC UIs, small hardware products, mobile service concept
• limitations: small areas of UI, experienced users, no in-depth thoughts, hard to express interaction

User-Centered Design for More Efficient Drill-Rig Control Systems by Katri Koli

• Leadin Inc (UX firm) working with Sandvik (mining equipment company)
• open-pit mine surface drilling equipment; drill holes, fill with explosives, blasting
• precise positioning of drill very important; 6 components, 2 directions of movement, 12 motions, traditionally use 2 joysticks
• develop automatic positioning mode; easier, faster, more accurate, user acceptance?
• method: contextual inquiry, iterative prototyping with simulator, usability testing
• study: 4 operative site visits; winter conditions; focus on hole positioning; 4 users of various experience
• challenges: restricted environments; recruiting participants through mine site; challenging environment (cabin designed for one operator, researchers behind operator chair, winter clothing even inside, may not be anything going on when there, safety prep, notebooks but possibly not photos or videos); getting enough interesting data (only 2-3 minutes of positioning in 60 minutes of work); working with simulator rather than real world for prototypes and testing
• collected 800 notes; need a clear research focus; affinity on all, but additional analysis on 1/4 that were about positioning; iterative prototyping and 2 rounds of 6 usability tests with drill rig simulator
• results: automatic positioning was faster, much more accurate, and easy to learn and use; products will ship this year; methods work with industrial users

Panel Question and Answer Session

• Q: would participatory methods work in the South Africa study? useful after the field visits when products were being explored
• Q: how were users compensated? mines: small gifts, cafés: coffee cards worth about $10, need to pay based on local culture and environment
• Q: did you run into situations where you weren't willing to work with individuals? Korea: hard to approach middle-aged men, mines: no issues, Africa: screener was actually a little too strict
• Q: did miners worry about effects of automation? increases safety and is more of a supervisory role so helped avoid uncomfortable work situations
• Q: how did you pick the right users? Africa: worked with local marketing firms
• Q: why two translators? difficult to translate directly, so played off each other and could also run errands and help deal with situations
• Q: how did you deal with being from a very different culture? working with locals very important
• Q: usability test didn't use the same operators? couldn't access actual operators but used company trainers who were familiar with work
• Q: did you have to consider non-standard conditions or failure conditions? have to be able to get out of full auto mode, still need to teach manual ways
• Q: how to avoid self reporting bias, an accurate baseline? mines: observe actual work in environment; cafe: many of our team are also app users so piloted with them;

Reasons to Be Cheerful – Part 4

• song: Reasons to Be Cheerful – Part 3
• HCI is good at thinking of other people's points of view, imagining ourselves as other people; eg users, maker communities; noticing, reflecting, questioning; everything seems to be speeding up, but we must take the time to think and reflect
• enjoyment comes from physiological needs met, strong relationships, meaningful work/activities, perspective and passion
• key directions: proactive health and well-being, marketplaces and exchanges, education and self-directed learning, data collection curation analytics experimentation interpretation, internet of things
• we in the HCI community have responsibility to keep people – as individuals and communities – in our technology systems; don't filter out the human emotions, empathy, culture, physiology, psychology; ensure that technology engenders and taps into joy
• known problems, known solutions; known problems, unknown solutions; unknown problems, unknown solutions
• Reflect: 5 things you found here that surprised you in a positive way; 4 new approaches or methods; 3 people you'd like to be in touch with; 2 sub-areas where you are out of your comfort zone you might influence; 1 grand challenge that you can engage in that may change the world

Customization Bias in Decsion Support Systems by Jacob Solomon

• user satisfaction improves with customizability; is it a good design choice for decision support systems?
• data -> system -> recommendation -> decision maker -> decision
• some systems support customization; customization -> recommendation quality -> decision quality; is this always true?
• customization bias: bias because decision maker has a part in driving the recommendation; reduce ability to evaluate quality of recommendation; supports confirmation bias
• experiment: fantasy baseball; predict scores assisted by DSS; one group could adjust statistical categories used, other couldn't; recommendations were predetermined, no algorithm, and both got same recommendations; subjects received 8 good recommendations and 4 poor recommendations; 99 MTurk participants with fair baseball knowledge
• findings: customizers had slightly better recommendations, but not the point of the study; customizers were more likely to agree with system; more likely to agree if recommendation was consistent with customization (confirmation bias); customization can enhance trust in system but trust is sometimes misplaced; ties decision making more to quality of recommendation (whether it gives good or poor ones)

Structured Labeling for Facilitating Concept Evolution in Machine Learning by Todd Kulesza

• data needs to be labeled for machine to distinguish; people don't always label consistently; concept evolution – mentally define and refine concept
• study: can we detect concept evolution; 9 experts, 200 pages, twice with 2 weeks in between; experts were only 81% consistent with prior labeling
• can we help people define and refine concept while labeling? added 'could be' choice to yes and no to allow additional refinement later after concept refined; often didn't name the groups, so then provided automated summaries; forgot what they did with a similar page, so automated recommending a group; not sure some pages were worth structuring, so show similar future pages
• study: 15 participants, 200 pages, 20 minutes, 3 simple categories; conditions of no structure, manual structure, and assisted structure
• findings: manual structuring created many more groups than automated; also mad many more adjustments in first half of experiment, less later; manual structuring more than tripled consistency and assisted almost tripled; took longer than baseline to label early items, but not longer for later items; preferred structured and assisted over baseline; easier to verify recommendation than to come up with their own

Choice-Based Preference Elicitation for Collaborative Filtering Recommender Systems by Benedikt Loepp

• recommendation system: select items from large set that match interests; collaborative filtering is most popular and is effective; criticized because focus is on only improving algorithms rather than improving user's role and satisfaction in use; also at beginning have no data to work from; ratings are inaccurate, comparisons are effective, but choosing comparisons depends on preexisting data
• goal: improve user effectiveness and control; generate a series of choices based on most important factors in a matrix factorization; items must be frequently rated, highly diverse choices, similar in non-choice factors
• evaluation: balance automatic recommendation and manual exploration; test 4 different user interfaces – popular, manual exploration, automatic recommendation, choice based model; 35 participants using each method to choose six movies + survey
• results: choice based significantly better than other models in all dimensions but required more effort than popular; good cost-benefit ratio; users felt in control; no profile or additional data required; works well for experience-based products

ARchitect: Finding Dependencies Between Actions Using the Crowd by Walter Lasecki

• activity recognition: system recognizing what you are doing; eg help people who may need assistance in living; automated systems need a lot of training data, where people can recognize very easily; crowd source from Legion:AR; still many permutations in behavior that must be recorded and labeled
• approach: define dependency structure to constrain meaningful variations
• ARchitect: ask.yes/no questions about different permutations of action steps to build valid models; eg 3 videos led to 22 valid models

Scalable Multi-label Annotation by Alex Berg

• multi-label annotation: identify aspects/objects that are or are not in an image; big in machine vision
• detect 200 categories in 100,000 images; large set is useful to many areas of research; expensive to scale, so exploit the hierarchical structure of concepts; correlation and sparsity; kind of like 20 questions for MTurk participants
• how to select the right questions: utility, cost, accuracy
• results: 20,000 images from set, 200 category labels; accuracy 99.5%+, 4-6x as fast