WEBVTT 1 00:00:03.350 --> 00:00:09.870 Michael Littman: Alright! Hi! Everybody thanks so much for for joining us. This is the the Nsf. Size 2 00:00:10.452 --> 00:00:18.318 Michael Littman: distinguished lecture series. My name is Michael Littman. I'm the division director for information and intelligent systems, and I'm hosting today's Speaker 3 00:00:18.760 --> 00:00:31.639 Michael Littman: Professor Jodi for Leezi, the Herbert A. Simon professor at at Carnegie Mellon University. I'm just really excited to get a chance to introduce her. She let me let me just review a little bit from her Bio, so that you have the the context. 4 00:00:32.130 --> 00:00:42.070 Michael Littman: As I said, the the Herbert A. Simon, Professor Herbert A. Simon, of course, being one of the founders of the Computer Science Department at Carnegie Mellon and one of the founders of the entire field of artificial intelligence. 5 00:00:43.400 --> 00:00:48.386 Michael Littman: and she has a professorship named for him, which is pretty cool. 6 00:00:48.850 --> 00:01:03.850 Michael Littman: She's a professor of computer science and human computer interaction in the school of Computer Science at Carnegie Mellon University. She's the faculty lead in responsible AI, which is a topic that everybody's thinking about all the way from individual researchers to the folks 7 00:01:04.409 --> 00:01:19.340 Michael Littman: in the seats of power in in the Us. Government and throughout the world, and and every individual person that I run into on the train, or the Metro, or wherever I am. People are thinking about this exact topic, and I think that's one of the reasons that we got such a great response with people tuning in for the talk 8 00:01:19.730 --> 00:01:23.648 Michael Littman: she but in addition to that topic, she is 9 00:01:24.390 --> 00:01:49.369 Michael Littman: in in the block center for technology and society, and she's also the associate Dean of Diversity, equity, and Inclusion, and in the school of Computer Science at Carnegie Mellon she's advocated for design research in all forms, mentoring peers, colleagues, and students in its structure and execution. And today it is an important part of the Hci community. And of course, it's very important part of the way that we think about building artificial intelligence systems in the years to come. 10 00:01:49.400 --> 00:02:12.439 Michael Littman: Jody studies the ethical impacts of human interaction with AI systems in frontline service industries, including healthcare and hospitality, which, of course, are getting a lot of attention in this topic, and she develops methods and tools to ensure that product developers can mitigate ethical harms and bias during product development, something that even in the areas that that Professor Felizi is not working indirectly can really benefit from this kind of perspective. 11 00:02:12.460 --> 00:02:16.129 Michael Littman: She recently testified to the Us. Senate Senate in one 12 00:02:16.565 --> 00:02:30.219 Michael Littman: an an AI innovation briefing and collaborates closely with Afl. CIO Tech Institute. So I'm going to turn it over to Professor Felici, and and again thank her very much for agreeing to present her work to us. We are all ears. 13 00:02:31.260 --> 00:02:51.960 Jodi Forlizzi: Thank you, Michael. Thanks for the nice introduction, and I'm very excited to be here. I was lost and travel delays yesterday, and got about 3 h of sleep, but I think my excitement is gonna carry me through the hour. I've been asked to speak for about an hour, which I feel is a bit long on zoom, so I'll do my best, and then we'll open it up for questions and answers. 14 00:02:52.310 --> 00:02:55.090 Jodi Forlizzi: So who am I? And why am I here? 15 00:02:55.310 --> 00:03:15.460 Jodi Forlizzi: So I I am often uncomfortable giving biographical information. But today I thought I would indulge you. I'm actually a first gen student. I'm the first person in my family to go to college, and I went to University of the Arts in Philadelphia, where I studied illustration, and that seems pretty strange because I'm in a computer science school now. 16 00:03:15.590 --> 00:03:29.839 Jodi Forlizzi: But after undergrad I worked at the University of Pennsylvania, doing technical illustrations and communications in an engineering and science building. And I think that was a forward of time, because I collaborate with scientists every day 17 00:03:30.300 --> 00:03:40.339 Jodi Forlizzi: with the advent of new technology that was really affecting design which at that time was the Internet. I decided to go to grad school to embrace the changes in my field. 18 00:03:40.480 --> 00:03:55.439 Jodi Forlizzi: And so I went to Carnegie Mellon and I was in the second class of the masters and interaction design at Cmu. And that was a really exciting time, because things were changing very rapidly. Computation was moving off the desktop and into everything. 19 00:03:55.770 --> 00:04:08.720 Jodi Forlizzi: I worked as a consultant after that at Elab, which was a small firm that did research for product design in Chicago, and that was also a really formative time, because it really raised questions for me about design research as a discipline. 20 00:04:08.750 --> 00:04:35.150 Jodi Forlizzi: And 2 years later I was asked if I would like to join the school computer science as the first design faculty member there. And since then I've had so many amazing collaborations and research opportunities. And I couldn't give this talk today without the support and collaboration of my faculty. My Phd. Students, masters, undergrads, Reus and my post stocks so too many to name. But thank you all. 21 00:04:36.320 --> 00:04:55.450 Jodi Forlizzi: And so today, I'm a design researcher. And I study the impacts of AI and automation in many forms, including assistive robots, automation and algorithms that track and manage workers. And today, I'm gonna start by talking a little bit about design and how it's changing and the research questions that we're asking response. 22 00:04:55.920 --> 00:05:08.290 Jodi Forlizzi: And Harold Nelson, who's one of my mentors, talked about accidental vagrants which are species that thrive in context where they don't traditionally belong. So, for example, land birds who live at sea. 23 00:05:08.550 --> 00:05:24.739 Jodi Forlizzi: all sorts of creatures can become accidental vagrants. They service creative and innovative solutions for positive change within an ecosystem. And so it is with me, I think, as a scholar of service and systems design working within human computer interaction. 24 00:05:25.490 --> 00:05:33.180 Jodi Forlizzi: And I wanna start my talk today with this notion that computing technologies, as we know, are changing the world at a rapid pace. 25 00:05:33.810 --> 00:05:51.529 Jodi Forlizzi: We are in the middle of a sea change in technology, starting with the personal computer way back when moving through. To continue with sensing VR AR. And more technology continues to evolve at a rapid pace, and a plethora of data has been generated. 26 00:05:51.750 --> 00:06:10.500 Jodi Forlizzi: People, of course, have benefited greatly from these developments, ranging from information transfer to crowd, sourcing speed of communications and beyond. But there also have been a lot of shortcomings. For example, hate speech, fake news, and the ability for people to operationalize on social media platforms 27 00:06:11.680 --> 00:06:23.119 Jodi Forlizzi: and data has been likened to the new oil and some writings and to the new electricity and others. It's true. Data has enabled new products, services, businesses, and economies. 28 00:06:23.560 --> 00:06:35.060 Jodi Forlizzi: Electricity changed, how the world operated it up ended every industry, transportation, manufacturer, agriculture, healthcare and AI is really having a similar impact. 29 00:06:35.170 --> 00:06:42.400 Jodi Forlizzi: Today it's hard to think of an industry that AI has not already transformed or is transforming. 30 00:06:44.120 --> 00:06:48.810 Jodi Forlizzi: And recently Sundar Pachai likened AI to electricity 31 00:06:48.990 --> 00:07:09.109 Jodi Forlizzi: and the center for security and emerging technology likened AI to fire. They said that it is much like how our ancestors encountered fire. If we manage it well, it can become a tremendous source for good. But if we deploy too quickly and without foresight, it will burn in ways we cannot control. 32 00:07:10.450 --> 00:07:16.340 Jodi Forlizzi: So back to my discipline, data, machine learning and AI are affecting everything that we design. 33 00:07:16.410 --> 00:07:29.789 Jodi Forlizzi: So, for example, AI can help to predict the risk of recidivism, offering efficiency in predicting whether a convicted individual will be likely to recommit an offense. 34 00:07:30.920 --> 00:07:44.920 Jodi Forlizzi: AI can improve educational outcomes for those at risk of not graduating by increasing the quality and quantity of feedback provided to students and teachers and suggesting new resources to advance teaching and learning. 35 00:07:45.950 --> 00:07:51.119 Jodi Forlizzi: AI is being used to advance health care through prediction and prevention of disease. 36 00:07:51.810 --> 00:08:03.489 Jodi Forlizzi: And AI can quickly recognize someone's image or voice and generate realistic replicas even with a small training sample which offers the ability to create a flood of 37 00:08:03.520 --> 00:08:07.070 Jodi Forlizzi: deepfake audio images and video tools. 38 00:08:08.760 --> 00:08:26.579 Jodi Forlizzi: And so today, machine learning and AI affect so many aspects of what designers touch and the tools that they work with ranging from web based environments to conversational uis to learning systems. And these technologies support a huge array of products and services. 39 00:08:29.110 --> 00:08:55.019 Jodi Forlizzi: So there's really 2 ways that we think about AI and design. One is using AI enabled tools, such as Photoshop or adobe products. And the design process. And the second is in designing AI or figuring out how to view AI as a design material like pixels or clay. And so that's really what I'm going to talk about today, because it's led to a broadening of the work of designers. And this has been evidenced in my own career. 40 00:08:55.910 --> 00:09:00.669 Jodi Forlizzi: So it may be helpful to provide some grounding ideas about what we're designing. Now. 41 00:09:01.030 --> 00:09:18.070 Jodi Forlizzi: as technology continues to advance and sensing and data become more abundant, it can be really confusing. So, for example, conversational Uis like Google, home or Amazon, Alexa can be thought of as products because they're physical things. But they're actually services. 42 00:09:19.830 --> 00:09:29.479 Jodi Forlizzi: So one thing, you one way you can test and see if something is a product is to ask yourself if it is a thing that is owned through the transfer of value 43 00:09:29.760 --> 00:09:37.730 Jodi Forlizzi: customers purchase products. There's a value exchange and transfer of ownership between a product maker and a customer. 44 00:09:37.860 --> 00:10:07.589 Jodi Forlizzi: and businesses get value in manufacturing products when they optimize production, reduce logistics, innovate on something technically or broaden the suite of product offerings. But this can get confusing, because software can be a product. And there's probably a few in this meeting that remember when Microsoft office, for example, was sold on a set of Cds. But these intangible products can be infinitely reproducible, and they can be pirated so they can also be disruptive. 45 00:10:08.570 --> 00:10:17.730 Jodi Forlizzi: Services, on the other hand, are not the same as products. They are intangible things that are enacted between a service provider and a customer. 46 00:10:17.780 --> 00:10:23.540 Jodi Forlizzi: and they require the presence of presence of a provider and a customer to take place. 47 00:10:24.470 --> 00:10:32.660 Jodi Forlizzi: Traditional service examples are things like banking, hospitality, healthcare and frontline services and retail and beauty. 48 00:10:33.180 --> 00:10:47.360 Jodi Forlizzi: Now notice that there are often many products within a service, and these form what I call product service ecologies. So, for example, Starbucks is a service, but the cup of coffee they provide is a product that the customer owns. 49 00:10:47.660 --> 00:10:54.140 Jodi Forlizzi: Starbucks is a service that delivers a product in a form and time and place that customers desire. 50 00:10:54.410 --> 00:10:58.870 Jodi Forlizzi: The performance is the fresh making and delivery of your customized drink. 51 00:10:59.670 --> 00:11:10.909 Jodi Forlizzi: So in many places in the world. You can go into a Starbucks, and you have this consistency of brand and product. You can get your lot latte or your cappuccino anywhere in the world. 52 00:11:11.220 --> 00:11:21.580 Jodi Forlizzi: For many people Starbucks serves as a third place between work and home, and they're also successfully driving commodities into other domains, such as homes and airplanes. 53 00:11:23.300 --> 00:11:28.789 Jodi Forlizzi: The world has pretty much transitioned from manufacturing products to providing services. 54 00:11:29.040 --> 00:11:55.969 Jodi Forlizzi: This New York Times article chronicles the shift in our economy from products to services. In the last 3 decades. On the left hand side you can see around 1960, the largest, most profitable companies in the Us. Were goods producing companies like GM. So and Westinghouse. But then, a few decades later, there was a sea change, and now the largest, most profitable companies in the Us. 55 00:11:56.400 --> 00:12:01.870 Jodi Forlizzi: Our service producing companies like Walmart, Target, Ibm, and Kelly. 56 00:12:02.090 --> 00:12:21.619 Jodi Forlizzi: And even though Target and Walmart sell commodities, we can think of their offerings as service offerings that are mechanisms to get customers, the goods that they need, and service design is needed to ensure that experiences are consistent and pragmatic and even ethical across all components of these systems. 57 00:12:23.210 --> 00:12:37.690 Jodi Forlizzi: So there is a benefit to a service design view, because it's a multi stakeholder view. And we just find a stakeholder is anyone with a stake in the Service Design project and anyone who's impacted by the Service design outcomes. 58 00:12:37.950 --> 00:12:56.970 Jodi Forlizzi: So if I give design students an assignment of creating a rideshare service. Usually on the first go around, they think of the driver and the passenger only they do not think of other impacted stakeholders, such as taxis, public transport, other cars on the road, and even cyclists and pedestrians. 59 00:12:57.380 --> 00:13:14.329 Jodi Forlizzi: Another interesting concept about serving is that it assumes that we are all on the same level rather than helping or fixing, which assumes that the people building the technology are better or higher status than the people receiving the solution. 60 00:13:15.820 --> 00:13:27.989 Jodi Forlizzi: Another grounding idea is this idea that products and services are blending and product service ecologies which you see at the top of the triangle come from efforts towards services and productization. 61 00:13:28.110 --> 00:13:38.069 Jodi Forlizzi: So servicesation is when we add services to products such as Ikea, who added home delivery and build services based on customer request 62 00:13:38.510 --> 00:13:54.749 Jodi Forlizzi: and productization is when we add products to service organizations. So, for example, Warby Parker started as an online store. But then it added physical stores to its online services to create welcoming customers, experiences around buying glasses. 63 00:13:57.030 --> 00:14:02.009 Jodi Forlizzi: Another construct is that of data or data animation. 64 00:14:02.140 --> 00:14:08.989 Jodi Forlizzi: So data can be added to products. This results in products like smart thermostats and water bottles. 65 00:14:09.120 --> 00:14:16.880 Jodi Forlizzi: So, for example, the Mattress company Casper sought to collect data around its mattresses with the goal of tracking sleep. 66 00:14:17.900 --> 00:14:34.450 Jodi Forlizzi: Data can be added to services. And we're seeing all this all the time in bountiful ways. We're seeing things, predictions being made about parts we might need when we go shopping or recommendations we might make for pairing wines and foods together. 67 00:14:34.980 --> 00:14:51.629 Jodi Forlizzi: And then, finally, data can be added to product service ecologies resulting in systems like Phillips, smart Sleep, which is comprised of wearables, apps on your phone environmental devices, and even an incentive problem for working on improving your sleep. 68 00:14:51.820 --> 00:15:04.029 Jodi Forlizzi: And so this is where it gets really interesting for designers, because we lack the processes and tools to readily innovate with all these aspects of these data enabled product service ecologies. 69 00:15:08.370 --> 00:15:11.100 Jodi Forlizzi: So another grounding idea is that 70 00:15:11.140 --> 00:15:15.259 Jodi Forlizzi: there is a difference between invention and innovation. 71 00:15:15.680 --> 00:15:27.379 Jodi Forlizzi: Typically scientists and engineers invent new technological capabilities, while designers and human scientists derive new innovative forms for these technical inventions. 72 00:15:28.000 --> 00:15:33.079 Jodi Forlizzi: So, for example, in 1,963, Phillips invented the cassette recorder. 73 00:15:33.370 --> 00:15:38.840 Jodi Forlizzi: they achieve this capability to record sound on a metal tape. 74 00:15:39.190 --> 00:15:47.280 Jodi Forlizzi: and there weren't really any applications at first. I think one of the first ones was probably the answering machine, the form factor which you see there. 75 00:15:47.330 --> 00:15:48.670 Jodi Forlizzi: But then 76 00:15:48.750 --> 00:16:00.399 Jodi Forlizzi: designers build on that capability in many forms, and we saw products like the Sony Walkman, the boom box, the double cassette recorder that enabled you to make mix tapes. 77 00:16:00.650 --> 00:16:18.140 Jodi Forlizzi: And some of these products really change social and cultural practices and society. I would argue that the Sony Walk band really changed. How we listen to music greatly. And so it's the advent of design building on these technical capabilities that gives rise to true innovation. 78 00:16:19.730 --> 00:16:29.569 Jodi Forlizzi: Now I would argue that an innovation gap exists with AI. This is something that we clearly show found in our research on AI innovation. 79 00:16:30.740 --> 00:16:46.049 Jodi Forlizzi: a 2022 survey by tech Republic reported that 97% of Ceos planned a significant investment in AI, but it also showed that about 80% of AI products that are developed fail before they go to market. 80 00:16:46.110 --> 00:16:51.299 Jodi Forlizzi: and of the remaining 20%, about 40% of those never succeed. 81 00:16:51.310 --> 00:16:56.810 Jodi Forlizzi: So there's a very small margin of products that actually make it successfully to market. 82 00:16:57.200 --> 00:17:03.740 Jodi Forlizzi: And our research in AI innovation has identified that AI products typically fail for 4 reasons. 83 00:17:04.250 --> 00:17:15.970 Jodi Forlizzi: First, model for performance teams cannot achieve the model performance. They need to have a good enough AI product. This might be because the data is unavailable or poor. 84 00:17:16.099 --> 00:17:19.970 Jodi Forlizzi: So if you can't get the data you need. You can't make good inferences. 85 00:17:20.020 --> 00:17:24.750 Jodi Forlizzi: Or maybe you can't get access to the data because it's hipaa protected. 86 00:17:25.650 --> 00:17:31.899 Jodi Forlizzi: Or maybe the data is old or biased or labeled poorly, all of these things can cause problems. 87 00:17:32.040 --> 00:17:36.550 Jodi Forlizzi: Second, when we generate a model, it can't make accurate predictions. 88 00:17:37.230 --> 00:17:41.469 Jodi Forlizzi: So if the model constantly makes the wrong predictions. The AI will fail. 89 00:17:42.490 --> 00:17:50.800 Jodi Forlizzi: and when teams choose a very hard problem with very hard AI. Often there's not going to be a success in a product development. 90 00:17:50.960 --> 00:17:52.590 Jodi Forlizzi: So, for example. 91 00:17:52.810 --> 00:18:06.959 Jodi Forlizzi: Galactica was met as attempt at a scientific, large language model for helping scientists. It was supposed to summarize paper, generate code, write math problems, generate Wikipedia articles 92 00:18:07.120 --> 00:18:22.380 Jodi Forlizzi: and more. But what it did instead was hallucinate greatly and create fake research papers, and I think it even said something about bears living on the moon. So Galactica, which had poor model performance, was taken down 3 days after it was launched. 93 00:18:24.470 --> 00:18:37.229 Jodi Forlizzi: The next reason why AI products often fail is that the service they provide don't give enough value back to the organization. And I would say that Amazon Alexa is an example of this kind of failure. 94 00:18:37.810 --> 00:18:50.660 Jodi Forlizzi: It's burned through tens of billions of dollars without ever really generating any revenue stream, because it seems like Amazon never tested the assumption that people would want to shop by speaking into their Alexa devices. 95 00:18:51.530 --> 00:19:01.139 Jodi Forlizzi: The third reason why AI products fail is that the service the AI provides does not give enough value to customers or users, and it's abandoned. 96 00:19:01.320 --> 00:19:10.120 Jodi Forlizzi: So a lot of times companies build sophisticated AI models to predict obvious things which are a waste of resources and time. 97 00:19:10.440 --> 00:19:13.849 Jodi Forlizzi: You don't need AI to predict that the weather's getting better. 98 00:19:14.060 --> 00:19:19.860 Jodi Forlizzi: You may not even need a chat, Bot, when all your company might really need is an easier to use website. 99 00:19:20.330 --> 00:19:45.959 Jodi Forlizzi: And then finally, AI products typically fail because they have ethical issues or privacy issues. And these are the ones we hear about the most often, and there are many, many examples of these, and there are public repositories where people add examples of these, and I think this really famous one was Twitters searching on Mitch Mcconnell versus Barack Obama, and it return pictures of 100 00:19:45.960 --> 00:19:52.920 Jodi Forlizzi: to Mitch Mcconnell's 2 white men. And this took on a whole storm in the Fate Research community. 101 00:19:54.930 --> 00:20:00.580 Jodi Forlizzi: So new methods and processes are needed to successfully and responsibly innovate with AI 102 00:20:01.010 --> 00:20:07.049 Jodi Forlizzi: in the design of AI products and services designers typically join the product development process. Too late 103 00:20:07.310 --> 00:20:10.870 Jodi Forlizzi: after data has been discovered and models have been built. 104 00:20:10.880 --> 00:20:22.070 Jodi Forlizzi: And so in our research on AI innovation, we've been developing methods to help designers understand AI as a design material and to help them more robustly generate and select 105 00:20:22.080 --> 00:20:30.390 Jodi Forlizzi: concepts early on in the process, and also to consider privacy and responsibility in the early stages of the innovation process. 106 00:20:32.250 --> 00:20:45.130 Jodi Forlizzi: So this means that traditional design processes may not apply. So anybody that's ever taken a design course look usually looks at the Double Diamond as the canonical model of User Center design. 107 00:20:45.370 --> 00:20:58.359 Jodi Forlizzi: And so with user center design designers start with a user or customer, usually developing things like personas and scenarios to elicit the problem and then suggest what an improved future state might be. 108 00:20:59.000 --> 00:21:16.369 Jodi Forlizzi: But when we're designing for AI, we can't start with the user. We actually have to start with the data set or capability and then work backwards to find the best customer. So this can be really hard for designers to grasp. So we've been working on some new methods and processes to get them to think this way. 109 00:21:17.570 --> 00:21:46.949 Jodi Forlizzi: Next, there are a range of skills in the technical portion of an AI development team. Of course, we need the canonical innovators at the top, the model builders who are creating the new algorithms and techniques as well as making existing algorithms more accurate and efficient. But we also need the engineers who develop custom models for specific applications. We need the developers who put the models into use. And we need the technicians who utilize the AI and potentially maintain it. 110 00:21:47.030 --> 00:21:55.970 Jodi Forlizzi: And all of these people have different and complimentary skills in working with AI and designers need to understand, communicate, and work with all of them. 111 00:21:56.810 --> 00:22:19.909 Jodi Forlizzi: So some of the tools we've developed in our research leverage familiar forms that designers use. So, for example, a service blueprint is a familiar model that serves as an orchestrating element for service designers. And we've annotated it with a data swim lane. So designers can start ideating around data possibilities early on in the project rather than users. 112 00:22:21.170 --> 00:22:36.119 Jodi Forlizzi: We also added a data source mechanism to a ux designer's wire framing process, really allowing them to figure out screen by screen. What is the underlying AI mechanism and capability for this product? 113 00:22:38.200 --> 00:22:44.950 Jodi Forlizzi: We've also refined this innovation approach called matchmaking. And it's really complementary to user-centered design. 114 00:22:45.000 --> 00:22:56.970 Jodi Forlizzi: So it asked designers or innovators to start with a specific data set or capability and then systematically search for the best customer application pair for the capability. 115 00:22:57.790 --> 00:23:08.320 Jodi Forlizzi: So technical capabilities often precede this identification of user needs. And one example I like to use is this filter from Snapchat Snapchat 116 00:23:08.530 --> 00:23:37.440 Jodi Forlizzi: AR developers created this method for a smartphone camera to identify facial markers when pointed at the user. But social media companies like Snapchat, recognized that this could provide a new type of shareable message. And so we have a plethora of dogs, pixies, unicorns, all kinds of things enabled, and this creation of face filters generated billions of dollars in revenue by increasing the use of social media services 117 00:23:39.540 --> 00:24:08.130 Jodi Forlizzi: in our Nsf. Grant on AI innovation, we created this taxonomy of AI capabilities and features. We started with a corpus, about a hundred current AI products and services and 40 features that cover a broad set of domains and a wide range of AI mechanisms. And this taxonomy really got us to the idea that the most successful design innovations often come from areas where moderate AI performance creates value. 118 00:24:08.130 --> 00:24:20.259 Jodi Forlizzi: So, for example, when you get a voicemail transcribed on your phone, it's really not a great transcription. There are errors, but it works well enough to let you know whether you should go in and listen to that voicemail. 119 00:24:21.180 --> 00:24:27.200 Jodi Forlizzi: It's not likely that customers would pay more for better transcription than this instance. 120 00:24:27.480 --> 00:24:37.480 Jodi Forlizzi: This is a great example of what we call the low hanging fruit of AI innovation places where moderately good AI is going to make a great acceptable product. 121 00:24:37.780 --> 00:24:45.349 Jodi Forlizzi: So this taxonomy serves as a mechanism for designers to envision concepts for future AI products and services. 122 00:24:46.680 --> 00:25:02.909 Jodi Forlizzi: We also designed this set of AI capabilities and features which can be used in an add on format to also help designers broadly explore different ways, that their applications might present the output of the AI systems to users. 123 00:25:02.910 --> 00:25:18.430 Jodi Forlizzi: so we could look at a capability like discover. And then, as designers, we could think about the kinds of discovery that could happen with AI, we could extract information, organize. We could group, cluster, connect, and reveal. 124 00:25:18.430 --> 00:25:29.789 Jodi Forlizzi: And then we could move to the right hand side and look at examples. And this begins to help novices who haven't innovated a lot with AI really begin to grasp its capabilities. 125 00:25:30.830 --> 00:25:37.040 Jodi Forlizzi: Similarly, privacy is also a key principle in developing successful AI products and services. 126 00:25:37.130 --> 00:25:50.420 Jodi Forlizzi: So our research on privacy by design, also funded by the Nsf. Explored, how AI capabilities and features can generate privacy risks, and we identified these 12 risks of a highest a result. 127 00:25:50.660 --> 00:26:02.209 Jodi Forlizzi: So this taxonomy can help product development teams think about risks of identification, disclosure, surveillance data, breaches and others. And this we see filling another gap 128 00:26:02.310 --> 00:26:12.590 Jodi Forlizzi: and current product development efforts that don't sufficiently address what kinds of privacy infringements happen, and how we inform customers about them. 129 00:26:14.760 --> 00:26:17.829 Jodi Forlizzi: and then, in terms of responsible. AI. 130 00:26:17.950 --> 00:26:25.380 Jodi Forlizzi: We are taking a broader look at the AI development process and looking for tools for governance in the AI pipeline. 131 00:26:25.480 --> 00:26:39.819 Jodi Forlizzi: We're attempting to to create a comprehensive overview of concrete mechanisms to operationalize responsible AI across the development pipeline looking at both the process. But all the stakeholders that might be involved. 132 00:26:41.250 --> 00:27:02.770 Jodi Forlizzi: So we created this stage by stakeholder matrix where we're working on articulating key tools and processes for each stakeholder during each phase of the AI development pipeline and color coding indicates the degree of influence and responsibility, and in doing this we have 2 matrices, one corresponding to issues and decisions. 133 00:27:02.770 --> 00:27:11.780 Jodi Forlizzi: and another corresponding to existing tools. As you know, this is a rapidly growing area. So I suspect that this will be a pretty dynamic 134 00:27:11.790 --> 00:27:14.109 Jodi Forlizzi: on matrix to keep updated 135 00:27:15.670 --> 00:27:31.869 Jodi Forlizzi: alright. So now that I've provided some grounding concepts around design and our research on AI innovation, I wanna talk about some of the application of these broader design skills and ideas and some of our research on AI and automation in the hospitality industry. 136 00:27:35.890 --> 00:27:45.110 Jodi Forlizzi: labor industry experts predict that technology has the potential to automate more tasks in the hospitality industry than any other industry. 137 00:27:45.630 --> 00:27:51.470 Jodi Forlizzi: Large numbers of hospitality workers are being displaced by these technological changes. 138 00:27:52.130 --> 00:27:58.680 Jodi Forlizzi: During the pandemic it was estimated that over 90% of hospitality workers were unemployed 139 00:27:58.830 --> 00:28:16.439 Jodi Forlizzi: and unemployment rates continue to be high, due to a shift in contact free services. If you've stayed in a hotel in the past year. You know that there's still a lot of touch, free, reduced service. No more room service, and this doesn't appear to be changing anytime. Soon 140 00:28:16.990 --> 00:28:28.220 Jodi Forlizzi: our research team is examining this change and studying the interventions and actions to help workers, employers and technology to be successful across this change. 141 00:28:29.590 --> 00:28:39.880 Jodi Forlizzi: So today in the hospitality industry. Robots meet, greet clean assistance, surveillance, transport, and deliver products and services. 142 00:28:40.580 --> 00:28:44.469 Jodi Forlizzi: so meet Flippy, Chippie, Sally, and Sippy. 143 00:28:44.500 --> 00:28:51.000 Jodi Forlizzi: They are robots that make food, deliver it to tables, clear tables, and serve customers. 144 00:28:52.250 --> 00:28:57.699 Jodi Forlizzi: But robots are only a small part of the technology used in the hospitality industry. 145 00:28:57.980 --> 00:29:13.100 Jodi Forlizzi: In addition to robots. We have AI VR and AR IoT smart rooms and smart sensors, big data and little data biometrics and facial recognition and conventional and conversational U eyes. 146 00:29:13.400 --> 00:29:22.710 Jodi Forlizzi: So these systems are used to track demand reservations, volume pricing. They're used to track inventory guests and employees. 147 00:29:22.800 --> 00:29:32.289 Jodi Forlizzi: And technology is really transforming the customer experience and the definition of face to face service which is traditionally associated with hospitality. 148 00:29:32.390 --> 00:29:45.410 Jodi Forlizzi: the numbers and types of jobs. But most critically, these technologies and systems are affecting the experience of the worker in terms of their tasks, their work flow, their work, volume, and skill requirements. 149 00:29:46.010 --> 00:30:04.849 Jodi Forlizzi: Many of these are hastily considered design, and there's even less consideration for implementation rollout training and impact to the workforce. So we like to say, an algorithm would never be implemented on a doctor without talking to them. But it's done all the time to hospitality workers. 150 00:30:06.180 --> 00:30:31.220 Jodi Forlizzi: So our work is seeking to create the opportunity for workers to envision design and engage in worker oriented research and training around the future of automation. And we've partnered with unite here the largest hospitality union in the Us. To ensure that worker satisfaction, voice, safety, ownership, and agency go hand in hand with an understanding of future technology and future work. 151 00:30:32.780 --> 00:30:46.120 Jodi Forlizzi: So I led the formation of a team of Hci design, hospitality, labor, and Hr. Experts along with members of unite here in the process of developing a proposal for the Nss. Future work 152 00:30:46.200 --> 00:30:54.510 Jodi Forlizzi: initiative. And in this work. We've built a really deep relationship with unite here, which is part of the Afl CIO. 153 00:30:54.750 --> 00:31:02.550 Jodi Forlizzi: And we really couldn't do this work without the research, because our Union collaborators have been with us in every step of the way. 154 00:31:03.010 --> 00:31:07.409 Jodi Forlizzi: and our project has 4 iterative phases that overlap 155 00:31:07.530 --> 00:31:14.989 Jodi Forlizzi: one was to understand the current state of union hospitality workers, hospitality work and automation technology. 156 00:31:15.160 --> 00:31:20.899 Jodi Forlizzi: Then to look at the iterative co-design of technology deployment models. 157 00:31:21.030 --> 00:31:40.180 Jodi Forlizzi: then to identify job skills and workforce needs and training materials to improve the current state and then to evaluate the outcomes, to see how they understand the impact and and impact the future work. And, by the way, this is a terrific funding program, and I hope it comes back. 158 00:31:41.290 --> 00:31:50.379 Jodi Forlizzi: So hospitality is invisible work. It is a physically demanding job done by women who are largely immigrants and women of color. 159 00:31:50.670 --> 00:32:01.810 Jodi Forlizzi: and as employers move away from daily housekeeping because of touch, free services, the work of housekeepers or guest room attendance, as they're also known, is becoming harder 160 00:32:02.150 --> 00:32:16.469 Jodi Forlizzi: in a checkout. The housekeeper does a deep clean of the room, changing sheets and towels, scrubbing the bathroom, etc, and then a stay over the work is much simpler. It's emptying the trash, making the bed, and maybe refreshing the toiletries. 161 00:32:17.180 --> 00:32:25.560 Jodi Forlizzi: In many hotels. Housekeepers are assigned to sections. Every day they clean the same floor or wing, or set of rooms. 162 00:32:25.930 --> 00:32:37.779 Jodi Forlizzi: and the assignment of rooms was traditionally done by hand by a housekeeping manager using a clipboard. And so the term of art for the list of rooms assigned still, today is called the board. 163 00:32:39.450 --> 00:32:44.229 Jodi Forlizzi: But digitalizing hospitality deprives housekeepers of autonomy. 164 00:32:44.610 --> 00:32:51.900 Jodi Forlizzi: When rooms are assigned algorithmically, housekeepers are told which room to do best next, based based on an algorithm. 165 00:32:52.100 --> 00:32:54.980 Jodi Forlizzi: Workers can't always see their whole board. 166 00:32:55.090 --> 00:33:06.659 Jodi Forlizzi: It requires tech literacy that interact with the system, and the board can change over the course of the day as different things get prioritized. However, there are also advantages. 167 00:33:07.270 --> 00:33:15.630 Jodi Forlizzi: Algorithmic managers and hospitality enable faster communication between housekeepers and other operations like managers and engineering. 168 00:33:17.490 --> 00:33:18.370 Jodi Forlizzi: But 169 00:33:18.650 --> 00:33:23.650 Jodi Forlizzi: the algorithm managers often assign rooms in an irrational oh 170 00:33:23.670 --> 00:33:41.439 Jodi Forlizzi: order! And this can lead to an extreme workload. So in this graphic, we can see on the left a comparison of how a human worker might move through a large multi wing hotel in the black line that would go from room to room, varying checkout and stay over rooms 171 00:33:41.900 --> 00:34:00.510 Jodi Forlizzi: as opposed to the algorithmic manager shown in the dotted line which has the housekeepers going up and down the hallway, up and down the floors, pushing a cart that is over 200 pounds and often doing many checkout rooms in a row. It's physically exhausting and nearly impossible. 172 00:34:01.990 --> 00:34:09.209 Jodi Forlizzi: So our team has been working on this problem since 2021. And this is a roadmap of what we've done to to date. 173 00:34:09.270 --> 00:34:14.479 Jodi Forlizzi: We've continued to synthesize the literature in this area which you see on the bottom. 174 00:34:14.610 --> 00:34:24.089 Jodi Forlizzi: We've conducted workshops and design sessions with guest room attendance. We've done observations and intercept interviews at 2 hotels. 175 00:34:25.188 --> 00:34:31.610 Jodi Forlizzi: And this is our portion of the work, and then our Hr. And labor partners are doing a 176 00:34:31.850 --> 00:34:42.779 Jodi Forlizzi: union steward or manager. Large interview survey, and we have some other things planned in the works which I'm gonna skip over in the interest of time. But this gives you a roadmap to date. 177 00:34:44.550 --> 00:35:02.299 Jodi Forlizzi: And so during the pandemic, when we first started this work, all our hopes of doing field work was shut down. So we ran online workshops with housekeepers, bartenders, and cocktail service servers focusing on their work what's changed with automation and what they prioritize and value. 178 00:35:02.740 --> 00:35:14.300 Jodi Forlizzi: So we started with housekeepers, bartenders and cocktail waitresses, exploring algorithm managers and co robots that are replacing some jobs and transforming others. 179 00:35:14.840 --> 00:35:19.760 Jodi Forlizzi: And these sessions strongly revealed workers' values in their hospitality work. 180 00:35:19.790 --> 00:35:30.679 Jodi Forlizzi: the desire to provide high touch, face to face services that customers desire, and that they take great pride in providing and providing, and they get tip force. 181 00:35:30.830 --> 00:35:38.199 Jodi Forlizzi: They have expertise and agency in doing their work as some have been doing it for over 25 years. In their current role. 182 00:35:38.900 --> 00:35:43.989 Jodi Forlizzi: They have a strong desire to collaborate with and if needed, help. Members of their team. 183 00:35:44.030 --> 00:35:49.210 Jodi Forlizzi: and they take pride and accountability and responsibility for a job well done. 184 00:35:49.770 --> 00:35:54.570 Jodi Forlizzi: But we also saw strong evidence of labor shifting in both technologies. 185 00:35:54.960 --> 00:36:09.760 Jodi Forlizzi: In both cases the automation systems created more labor for certain hospitality workers and reduced their ability to perform the important and desirable social and emotional labor of making connections with customers. 186 00:36:10.160 --> 00:36:26.359 Jodi Forlizzi: So, for example, in casinos that have augmented bartenders with these robots or co-robots that make free drinks, cocktail waitresses describe the poor quality of the drinks, the need to shuffle, drink orders to prevent contamination. 187 00:36:26.460 --> 00:36:32.650 Jodi Forlizzi: and the idea that they couldn't complete their rounds as quickly due to serial interactions with the machine 188 00:36:33.120 --> 00:36:50.710 Jodi Forlizzi: hospitality workers express frustration and a lack of training and using automation system in new and greater and positions of management on the agency with which they formally did their job and the need to constantly mitigate the negative impacts of technology on customer service. 189 00:36:52.600 --> 00:37:01.309 Jodi Forlizzi: So the social science literature on automation has traditionally focused on job displacement as seen in industries like manufacture. 190 00:37:01.820 --> 00:37:09.489 Jodi Forlizzi: And there's not much empirical evidence in industries like housekeeping and other face to face service. 191 00:37:09.760 --> 00:37:14.569 Jodi Forlizzi: So the recent literature highlights more of a complexity in this sector. 192 00:37:15.530 --> 00:37:22.699 Jodi Forlizzi: specific skills are challenging to automate and work process and job content change. 193 00:37:22.870 --> 00:37:26.759 Jodi Forlizzi: So as technology transforms the experience of work. 194 00:37:26.970 --> 00:37:29.610 Jodi Forlizzi: The organization also plays a role. 195 00:37:29.800 --> 00:37:40.140 Jodi Forlizzi: And it's also important to note that in the hospitality industry it's even more complex because a hotel can be owned, branded, and managed by up to 3 different entities. 196 00:37:40.270 --> 00:37:58.950 Jodi Forlizzi: So this really affects how technology is rolled out in a particular property, and it can really affect whether basic minimal effort, low road implementation takes place, or if a change management team is utilized to manage high road robust implementation. 197 00:38:00.590 --> 00:38:05.880 Jodi Forlizzi: So I want to touch on a few ways that design, research, and design methods are at play in this work. 198 00:38:06.180 --> 00:38:14.609 Jodi Forlizzi: and they include principles from service design and practices, from design methods that ensure that we are taking the right steps. 199 00:38:16.180 --> 00:38:24.680 Jodi Forlizzi: So so first, I wanna just introduce systems, design and systems thinking, this is a larger practice than service design. 200 00:38:24.930 --> 00:38:27.999 Jodi Forlizzi: A system is the sum of its parts. 201 00:38:28.010 --> 00:38:32.650 Jodi Forlizzi: But really, we're looking at the product of the interactions between those parts 202 00:38:33.350 --> 00:38:39.019 Jodi Forlizzi: and in complex systems, such as those as I've described in the beginning of my talk. 203 00:38:39.030 --> 00:39:07.690 Jodi Forlizzi: we're really designing more than services. We're designing systems and complex Sis situations. And Harold Nelson would talk about this as more than creative problem solving and more than design thinking and Nelson use this metaphor of charting the course of a boat rather than managing its wake. So we wanna consider all the components in the system holistically to try to get to the next steps in the research. 204 00:39:08.320 --> 00:39:36.019 Jodi Forlizzi: So often we use this iceberg model or iceberg metaphor from systemic design to note that the events that we see in our observations are just the tipping the iceberg. So as we analyze that current state, we have to understand the system around the problematic situation, and then the behaviors, the trends, the structures, and even the mental models that are driving these things. 205 00:39:38.460 --> 00:40:02.070 Jodi Forlizzi: So in this work, it's really critical to surface the voices of multiple stakeholders. So talking to the housekeepers gave one perspective, talking to the housekeeping managers, gave another perspective. Talking to software manufacturers which we've been able to do revealed another perspective, sometimes conflicting and talking to hotel management revealed yet deeper and additional perspectives. 206 00:40:02.700 --> 00:40:25.879 Jodi Forlizzi: And sometimes there's this prevailing notion that tech that the technology is bad. But in many cases errors and misuses just occur from misunderstandings, uneven training, inconsistent configuration of the software, etc. And by looking at things holistically and with a multi disciplinary team we can really begin to get at these problems. 207 00:40:27.630 --> 00:40:49.219 Jodi Forlizzi: So a lot of times we use this design skill of reframing and I would say in our work, it's really important to reframe this idea. That tech is bad, because if we keep that framing we really won't be able to advance in our research. So this notion of reframing which is basically getting to the assumption behind the assumption can be useful. 208 00:40:49.420 --> 00:40:55.099 Jodi Forlizzi: So, for example, we could compare Amazon delivery workers and ups delivery workers. 209 00:40:55.550 --> 00:41:07.029 Jodi Forlizzi: These workers are both monitored in their jobs, but they're actually treated quite differently. Ups workers are viewed as valuable data collectors who improve algorithmic systems with their work. 210 00:41:07.140 --> 00:41:13.310 Jodi Forlizzi: But Amazon drivers seem to be managed by algorithms and viewed as expendable commodities. 211 00:41:13.780 --> 00:41:25.060 Jodi Forlizzi: There was a case of a driver in Arizona who had worked for Amazon for 4 years, and claimed that he was fired by an algorithm that ascertained that he was not doing his job properly. 212 00:41:25.640 --> 00:41:28.950 Jodi Forlizzi: Apparently he was punished for things beyond his control 213 00:41:29.140 --> 00:41:32.039 Jodi Forlizzi: that prevented him from making his deliveries. 214 00:41:32.440 --> 00:41:37.669 Jodi Forlizzi: And so this algorithmic management, with little human oversight, appears to be problematic. 215 00:41:38.160 --> 00:41:43.170 Jodi Forlizzi: On the other hand, ups, empowers and informs its workers. Substantially more. 216 00:41:43.380 --> 00:41:57.999 Jodi Forlizzi: The data collected about ups drivers is reviewed by humans, not algorithms and ups drivers cannot be fired by an algorithm. And then if they do get fired, they're unionized. So they have a process in place for appealing it. 217 00:42:00.060 --> 00:42:11.769 Jodi Forlizzi: So, returning to our research, we often use prototypes and other design technique to help imagine an ideal future along with the social and cultural norms that might surround it. 218 00:42:12.080 --> 00:42:31.440 Jodi Forlizzi: So, as you might have gathered in our initial workshops. We heard a lot that the agency of the housekeeper seemed to be lost when they are working with an algorithm manager. So we conducted a second round of workshops and interviews to understand how these algorithmic managers actually impact daily work. 219 00:42:32.080 --> 00:42:52.970 Jodi Forlizzi: Then researchers collaborated with the worker participants to sketch new designs, and we emerge with this idea that with some of these designs for inch, for instance, being able to self sequence or order, room assignments, workers could feel more self efficacy. 220 00:42:52.990 --> 00:42:55.800 Jodi Forlizzi: experience more transparency. 221 00:42:55.810 --> 00:43:00.199 Jodi Forlizzi: better collaborate with their team and reduce their workload overall. 222 00:43:01.600 --> 00:43:22.399 Jodi Forlizzi: And this was mirrored by what was going on in the real world, because about the same time the Las Vegas culinary union local 2, 2, 6 actually gathered together and gathered issues around self sequencing and went to their stewards, asking for language in their contract around the ability to self sequence rooms 223 00:43:25.350 --> 00:43:50.770 Jodi Forlizzi: in the next steps of our research. We want to dig deeper into where self sequencing, where AI, where working better with systems and having better digital literacy, might serve these workers. So in a first of a kind collaboration between a culinary training academy, our research team and a software developer of these algorithmic managers for hospitality. 224 00:43:50.770 --> 00:44:05.870 Jodi Forlizzi: We will do a study where we are exploring modifications to the software design with the skin of their software to look at how we might enable soft self sequencing, increase digital literacy and increase AI literacy training. 225 00:44:08.990 --> 00:44:14.799 Jodi Forlizzi: So our approach in this product project has had some even greater and more exciting help 226 00:44:14.870 --> 00:44:26.479 Jodi Forlizzi: outcomes. We have worked at Carnegie Mellon University to fulfill a larger obligation to create opportunities for workers and labor organizations 227 00:44:26.580 --> 00:44:31.890 Jodi Forlizzi: to participate in this kind of research and improve outcomes for society. 228 00:44:32.610 --> 00:44:42.460 Jodi Forlizzi: There is a growing consensus among policy makers that innovation and technology development are strategic imperatives for our country. 229 00:44:43.060 --> 00:44:54.929 Jodi Forlizzi: And recently the President signed the Chips and Science Act to establish a technology strategy with guidance from Nsf. Directory on the new tip partnership. 230 00:44:55.140 --> 00:45:06.579 Jodi Forlizzi: And so this is a really exciting collaboration that brings together universities, industry, government and labor organizations as 4 critical stakeholders in this directorate. 231 00:45:06.610 --> 00:45:12.050 Jodi Forlizzi: And so it's really exciting to see our research as being formative in these ideas. 232 00:45:12.871 --> 00:45:21.060 Jodi Forlizzi: And helping to develop this partnership with the Afl CIO and the Tech Institute and 58 affiliate labor unions. 233 00:45:21.320 --> 00:45:32.650 Jodi Forlizzi: because all of us really want to have technology be doing the right things and to be servicing work holder, worker, stakeholder voices in its development, deployment and use. 234 00:45:33.250 --> 00:45:37.360 Jodi Forlizzi: And so the Afl CIO Tech Institute has. 235 00:45:37.550 --> 00:45:52.329 Jodi Forlizzi: I said, is a goal. This bringing worker voice into the innovation process with the actions of improving technology development through worker engagement and collaboration and providing equitable access to jobs and career paths 236 00:45:52.540 --> 00:46:03.179 Jodi Forlizzi: by bringing worker voice and labor organizations into the innovation process and by creating a model of labor academic engagement across sectors and communities. 237 00:46:05.300 --> 00:46:14.999 Jodi Forlizzi: And we've also had the benefit of testifying to Senator Schumer Rounds Heinrich and Young in a closed door, briefing on AI innovation. 238 00:46:15.420 --> 00:46:30.219 Jodi Forlizzi: and my remarks were based in part on this work in the future of work, grant and I called for 3 recommendations. The first was to bring workers into the design, development training and deployment process of AI systems 239 00:46:30.410 --> 00:46:54.330 Jodi Forlizzi: and to design sustainable AI from the very beginning of the product development process, figuring out what's idiosyncratic and what's generalizable. And third, to make digital and AI literacy training programs readily available for all workers. And these ideas fed into some of the language in the executive order released later that month, which was October 2023. 240 00:46:55.930 --> 00:47:05.670 Jodi Forlizzi: So, in conclusion, I wanted to reiterate some of my points in my talk today. First designers are moving from product to service and systems design. 241 00:47:06.080 --> 00:47:10.309 Jodi Forlizzi: second, products and services are blending and are data animated. 242 00:47:10.490 --> 00:47:14.140 Jodi Forlizzi: Third, designing with AI and machine learning is different. 243 00:47:14.270 --> 00:47:20.259 Jodi Forlizzi: and force that the role of design is broadening in the AI research and design processes. 244 00:47:21.010 --> 00:47:31.849 Jodi Forlizzi: Finally, we need a broader systemic investigation of technology in service domains and reframing, getting to the assumption beyond the assumption is critical. 245 00:47:33.060 --> 00:47:36.640 Jodi Forlizzi: So I want to say, thank you again for listening. 246 00:47:36.740 --> 00:47:49.740 Jodi Forlizzi: It's really exciting to see how design, design, research, and design practice continue to involve along with technology and are is now being applied to more complex problems 247 00:47:49.740 --> 00:48:09.830 Jodi Forlizzi: and being able to bring more benefits to society. So I wanna thank the Nsf. For the opportunity to give this talk, and for all your generous funding, and again to my students from undergrad re use all the way up to my post docs and my faculty collaborators for being able to do this work. Thank you very much. 248 00:48:15.080 --> 00:48:16.730 Michael Littman: Thank you so so much. I I.