Okay, getting ready to start the Webinar now,
Michael Littman: all right. It looks like maybe the numbers are leveling off a little bit. So so I think I think we'll get started. Welcome, everyone. Thanks. Thanks a lot for coming. This is the the Distinguished Lecture series for the Directorate and and Nsf. Of computer information, science and engineering.
Michael Littman: Uh, led by Margaret Montagnosi. My name is Michael Litman. I'm. One of the
Michael Littman: for division directors in the Directorate. My own area is in artificial intelligence and machine learning, and my directorate is called Iis Information and Intelligence Systems. And if you, if you look on the web which I just did to see what it is that my division does.
Michael Littman: It says Information and intelligence systems, studies the interrelated roles of people, computers and information to increase the ability to understand data as well as mimic the hallmarks of intelligence in computational systems. And so i'm really excited about today's speaker, Fernando Vas
Michael Littman: from from Harvard University. She Well, you can see from the title harnessing the power of data, visualization, from insight and storytelling to ai explanation and data art. So she is working right at that boundary between people and data, information and computation, and when we were soliciting within the division for for possible speakers, and and her name came up, the The response was overwhelming. Her work is just beloved here in our division, and i'm very, very excited to hear
Michael Littman: she's got to say on on these topics. So Um. Oh, and I also I forgot to mention this sooner. I probably should have mentioned this different and sooner is that we'd like, if you could, to say a little bit about
Michael Littman: your own journey, how you came to be a researcher and working on these things. This is something that we've done with all our distinguished lecture Guests and people really resonate with that. It's a great way of of helping them see themselves in this kind of work that goes on. Scientists can be, of course, very, you know,
Michael Littman: precise and official, and but we're really all people, and I think, Fernando, I have no no question. There's no doubt that you would get into that, especially using words like data. Art. You're very much in touch with your human side. But if you can tell us a little bit about yourself, that would be that would be lovely. So, ladies and gentlemen, Ah, Fernando be, I guess,
Fernanda Viégas: Thank you so much. Thank you for the first of all. Thanks for the invitation. Thanks for having me here. I was very much. Ah! I was very excited, and looking forward to this and Michael, i'm so glad that you asked about my career, c. Because I even have a slide about that. So I definitely do not have a traditional path to where I am at today. In fact, I would never have guessed that I would do what I do today. I am from Brazil.
Fernanda Viégas: I grew up in Rio, and I actually, when it came time to decide what Major what university
Fernanda Viégas: I had a huge, huge. It was a very hard time. I could not decide what I wanted to do. And in Brazil, if you change it, if you going to university for say computer science or something, and then you change your mind and you decide you want to do something else. You have to leave the university and try to get in again for another major.
Fernanda Viégas: I did this three times, and three times I failed to stick to whatever major I was going.
Fernanda Viégas: I was like I don't think University is for me, and then I got a scholarship to come to the Us. The main reason I came to the Us is because it could be an undecided major here. So that is the whole point of having come to the Us. I thought that was an amazing idea. I ended up going to the University of Kansas. My background is in graphic design and art history. So again, nothing to do with technology.
Fernanda Viégas: But then I decided, when I was about to graduate from graphic design and art history.
Fernanda Viégas: I started looking around, and I became familiar with the Media lab at Mit, and I noticed that they were welcoming people from different backgrounds, graphic design included. So that's where I ended up doing my graduate studies.
And that's where I learned about data visualization. And to me my mind data visualization is a great,
Fernanda Viégas: you know, connection between graphic design and computation. So I had to learn how to program. I had to. It was really hard, but I I loved the field Um! And then from two thousand and three on, I started collaborating with Martin Watenberg, who is a mathematician by training and everything you're going to see today that I'm going to talk about is work that I've done with Mark. We've been working together for almost twenty years now. So it's
Fernanda Viégas: It's been great, all right. So now let's jump in. I want to give a lot of demos, and then I want to give a broad overview of the field of data visualization.
Fernanda Viégas: So let's start with this little project, Martin and I did many years ago. It's called webs here. This was before we joined Google. We became very interested in the fact that when you start typing something on Google,
Fernanda Viégas: Google gives you a bunch of suggestions to finish whatever text string you've started. So here I typed Will Brazil, and then Google gave me a bunch of suggestions. And Google does that for utilitarian reasons, it says, Oh, if you're saying, will Brazil,
Fernanda Viégas: chances are you're going to ask, Will Brazil, when the World Cup or something like that right now, right, and it's trying to save you time. So it's trying. It's showing you the most popular endings to your string.
Fernanda Viégas: But Martin and I took a look at this, and we thought, Oh, wait a second. This is not only utilitarian, it's also a little peek into the public Psyche right? It's what people come to Google, for so can we visualize this. So this is what we we did. We created a little visualization, and here i'm going to
Fernanda Viégas: show you how it works, I can say. Will Brazil. Oh,
Fernanda Viégas: if I spell it correctly, will Brazil, and then I am literally just visualizing the same stream of completions that Bobo was giving me. But now, because I am
Fernanda Viégas: visualizing this, I can start playing little games, so I can compare the connections between Will Brazil to the
Fernanda Viégas: will the Us. And boom. I have the places where they come together, so people are very interested. If, whether Brazil or the Us. Will win the World cup,
Fernanda Viégas: they are also so, which is very hopeful. But then another completion that is the same between the two of them is kind of the splendid. Will Will there be a civil war in either one of these countries? Um. And you can also see how the different completions go for each one of the countries. So you start to see It's kind of like a van diagram, if you will, of these things. Um. So this is very World Cup related sometimes, but because of the moment we're in. But
Fernanda Viégas: I can do things like Why doesn't he versus why doesn't she?
Fernanda Viégas: And you can see what people are curious about? Will he text me? Will she love me? And so forth? I can also do things like is my son versus is my daughter,
Fernanda Viégas: and you can see what parents are coming to Google to to ask, and this is both interesting. But to me it's also kind of a a gut punch. It's, you know people are feeling quite vulnerable. They are asking vulnerable questions to this search engine right? And I think that's that's interesting. And one of the other things that I think data visualization does that you can start to see? Hopefully, here
Fernanda Viégas: is that
Fernanda Viégas: to me a lot of times it makes data seem less
Fernanda Viégas: cold and statistical and official, and more humane and kind of like. Oh, wow! Really, people are coming here with these questions, and you can start to put yourself in their shoes. So is my husband versus is my wife.
Fernanda Viégas: Um, and you can. You know people are wonder if their partners are cheating on them, or attracted to them, or artistic, and and so forth.
Fernanda Viégas: Don't, worry not. Everything is negative or a downer. You can also say you can also ask if coffee versus is chocolate.
Fernanda Viégas: Two things I really love. So it's coffee good for you. It's chocolate and good for it. And what is up with the dogs? Right? It's it's interesting. Actually, let's go. There
Fernanda Viégas: can say cats are versus
Fernanda Viégas: dogs are, and the good news is that everybody agrees they're all the best, which is very good and positive. So I will leave on this note here. But this is a live public demo. Anyone can play with us.
Fernanda Viégas: Um: Okay.
Fernanda Viégas: One of the reasons why I wanted to start with. This is the fact that sometimes there is this notion that data visualization is mainly for visualizing numbers, and it and a lot of it is right, and it's very useful that way.
Fernanda Viégas: But I think one of the things that we now have the ability to do is to visualize a lot of different kinds of data text being one of them. And so you're going to see some of that today in this talk, so. Part of the points I want to make here is that visualization
Fernanda Viégas: is not just for numbers,
Fernanda Viégas: and it's not just for individuals. Usually we think about one person in front of a computer.
Fernanda Viégas: It's it's not like that. Actually, visualization tends to be a very social thing, and it's not just for experts. For the longest time. Visualizations were built by experts for experts, and I think today we're moving beyond this, which is very exciting, because it helps with things like
Fernanda Viégas: number, literacy and statistics, statistics, literacy for society at large. So we'll talk a little bit about that.
Fernanda Viégas: I also want to take a step back and say, Ah, a lot of what you're going to see here today, and a lot of what we do in data visualization we've been doing for a while, Truth is, and so like. We've been mapping things for a very, very long time. So like these are some of the earliest maps we have, you know. Ah, notice of the the one on the on the left here. So, being a Babylonian map carved the one on the right. Very interesting is a Ptolemaic
Fernanda Viégas: world map from the second century. And the really cool thing about this map is that it's the first time we see the use of longitude and latitude. So they came up with this concept. They came up with this standard of mapping things around the globe, and we use it to this day. And I think these are really interesting, you know. First first offs,
Fernanda Viégas: when we move away from like
Fernanda Viégas: things that have shapes. So, for instance, if I go back to the maps. These things have shapes in the physical world
Fernanda Viégas: around us, right when we move from, though from those things to the
Fernanda Viégas: information, abstract information that really doesn't have a shape. How can we visualize those things that are invisible? And this is where people like William Playfair comes in. So this is literally one of the first. He invented the line chart,
Fernanda Viégas: and he also invented the pie chart and the bar chart. I mean, how crazy is that! How amazing is that that the same person invented all of these techniques to visualize data. We have someone like Florence Nightingale who also invented it. She invented this this kind of we call it a rose diagram. And
Fernanda Viégas: to me the really interesting thing here is that she was both a nurse and a statistician, and she used charts for activism. The whole point for creating this chart you see on the left, which, by the way,
Fernanda Viégas: shows the death of soldiers in the Crimea and Crimian Crimea and War Um. She was trying to make the point that by far most soldiers were dying of wounds in the hospital, not in the battlefield.
Fernanda Viégas: And And so all of the blue. You see, there are deaths at the hospital, and the red are deaths in the battlefield, and you can see just how much more. And so the point that she was trying to make is, we need to change the way our hospitals work. And so this chart
Fernanda Viégas: um convinced the British Parliament to enact sanitation reforms in hospitals, and thank you for those we have. We do things today to this day? Um, in better ways, because uh, because of the kind of data and visualization she created which is wonderful. Um, um, Webb du boy. So he was really interested in visualizing the
Fernanda Viégas: condition of uh black Americans after slavery, he was saying, Look, we're not slaves anymore. But this is not working people. And so he created a whole series. He and a team of sociologists created a whole series of charts that are quite
Fernanda Viégas: aesthetically striking about the condition. Um, in society of black Americans so very influential, So all of this is to say, this is where a lot of the work that you're going to see today comes from. It has a lot of historical context.
Fernanda Viégas: Okay, So not just for numbers. Let's go back to that idea that I talked about in the beginning. Um, One of the things that you may think about when you, when you think about visualizing text immediately, are like tag clouds or word clouds, and and those are fine, but they take away a lot of the context
Fernanda Viégas: of what is being said. So one of the things that Martin and I were interested in. How can we visualize trends in text, but still keep context available. So we created the word treat. Imagine you have something like Romeo and Julia the entire play, and you do a search for a string, like, if love.
Fernanda Viégas: What I want to do after this is find all the completion. So it's kind of like the Google thing, all the completions that come after, if love. So if love be rough with you, or if you love to be blind, all of these things. Now I want to visualize those completion. So i'm going to do kind of a tree. I'm going. It's kind of like a Suffolk street,
Fernanda Viégas: and I will show you the big trends, but also the context. I will show you all the sentences at once. And so, for instance, I have a dream. The speech by Martin Luther King. We can visualize it as a tree to understand all the different places in the speech that that phrase comes up.
Fernanda Viégas: Um! This is a visualization of the Bible. And so what are all of the places in the Bible where love shows up?
Fernanda Viégas: Um, and I will show you a very quick little demo. This is. This was a word tree iteration done by Jason Davies, based on the algorithm that Martin and I created and it's available online. Anyone can play with this, and we're going to look at very quickly at Steve Jobs commencement speech at Stanford.
Fernanda Viégas: So here, what i'm doing is, and you can see that this is interactive. I have looked for the string life,
Fernanda Viégas: but one of the things I've done is to reverse the treat. I want everything that ends in life,
Fernanda Viégas: and I can now click on, for instance, my life. And now i'm zooming in into that part, that branch of the tree, and I can keep doing this. I am going to reverse this and say just my
Fernanda Viégas: and you can see all the places in his Commencement speech where the word mine comes up, and he's he's talking about his life. He's talking about his parents, his biological mother. He's even talking about his cancer um right. And so there are one of the things that to me is really powerful about this is that it gives you a very fast way of navigating a large body of text without again losing the context of what is being said.
Fernanda Viégas: So again, things, you know, ideas around visualizing hard data sets like like text.
Fernanda Viégas: Another way of visualizing
Fernanda Viégas: activity around text is
Fernanda Viégas: a project Martin and I did back in two thousand and three. In fact, this is the very first project I ever worked on with Martin, and this is what it is. We were interested back then. Nobody understood Wikis
Fernanda Viégas: and everybody was skeptical of Wikipedia. They're like. How can this work? This doesn't make any sense. Anyone can edit this thing.
Fernanda Viégas: And so we got very interested in understanding the dynamics. How are people collaborating around Wikipedia articles?
Fernanda Viégas: This is a vintage screenshot of an article from two thousand and three that happens to be on chocolate again. One of the things I really love,
Fernanda Viégas: and we learned that behind each article
Fernanda Viégas: you have kind of an activity, a history log of all the Times. That article had been edited, and we decided, this is what we want to visualize. We want to visualize. It has timestamps, it has. Who did what? And so this is the data we decided to visualize. We created a technique to visualize this.
Fernanda Viégas: So imagine you have three people who are going to work together on an article Mary Suzanne and Martin and I'm going to give a different color for each one of those people,
Fernanda Viégas: and I'm. Going to color each version by whoever was active in that version. So version one was all written by Mary.
Fernanda Viégas: The length of the line on version one is the length of the article,
Fernanda Viégas: and then you can see that in version two
Fernanda Viégas: that draft that that article stays. But there is a little blue line at the bottom. That's the paragraph that Suzanne added to the end of the article, and then on version two. It gets shrunk again because Martin came and deleted a piece of the initial
Fernanda Viégas: orange line and insert it a little. You know a little piece of text. So and this goes on. And to make these things even more clear to see, we connect all the text that survives. Okay. So you can start to see holes and and things being growing and and shrinking.
Fernanda Viégas: Another thing we can do now that we have this visualization is to actually visualize the same data in real time. So I can now start to see rhythms. So oh, yeah, between version one and version two a long time,
Fernanda Viégas: you know it took a long time, but then version three came up right after. I can also highlight different versions of this text, and I can actually see the
Fernanda Viégas: the raw text being displayed with the color of the author. So now let's actually see a demo of this. So this is the project is called history flow,
Fernanda Viégas: and you can see a big diagram in the middle. This is the evolution of the article on Design on Wikipedia on the left. Here I have all the people who have edited this article on the right. I have the
Fernanda Viégas: the article itself, and I also have a little wand that I can move around and I can see the article changing over time.
Fernanda Viégas: Okay,
Fernanda Viégas: Nothing to
Fernanda Viégas: different. Oh, i'm going to need access to a little
Fernanda Viégas: drop-down menu.
Fernanda Viégas: Okay. Now let's look at a different article. So the article on cats. So first off
Fernanda Viégas: many more people like cats than design on the Internet. I'm not surprised. Um, and it's a long article. It's a long article, and there are interesting things going on here. One is this stripy pattern here, and what this is. I can go back here to the beginning of the pattern, and I can see that someone added a table of the kingdom and the class and the order and the family of cats, and this survives. I can see that it's arise forever. So that's great.
Fernanda Viégas: There's one one thing that's different. Here there's this antenna at the bottom that doesn't go anywhere what's going on there? So if I come over here I can see that someone added a whole bunch of paragraphs about the unix command cat,
Fernanda Viégas: and right so for those of you who know that command It's funny right. It's like, Oh, does that fit in the page about cats on Wikipedia, or does it not because it's gone in the next in the next
Fernanda Viégas: edit. So it's in the next version. So if I go there I actually see that it's not completely gone. Someone just created a new page called Cat Unix, and redirected all of that content there. So we were starting to see how are people? What is the dynamics of of collaboration here?
Fernanda Viégas: I'll show you a different page. So we're going to see the page on abortion now and again lots of people
Fernanda Viégas: contributing to this a very long article, but there are a couple of interesting things here. There are a couple of gashes that happen here in the middle of the diagram. So if I come over here I can see that the entire article was deleted. So this is vandalism. It's called mass dimension. If I come over here I can see that not only someone deleted it, it said. Someone said, Abortion is great, and then they added, Abortion is good, and then it got reverted.
Fernanda Viégas: Okay. So vandalism exists in Wikipedia.
Fernanda Viégas: But here's the thing that's interesting. If we look at when this happened the very first vandalism count. It happened on December the seventeenth, at four hundred and six,
Fernanda Viégas: and it got reverted on the same day at four hundred and seven. So it took a minute for them to solve this,
Fernanda Viégas: and we kept seeing this over and over again where we were like. How are they doing this, so we got in touch with Wikipedians, and we learned about something called the Watch page. You can have a watch list of all the articles you care about, and you get notifications when someone makes an edit.
Fernanda Viégas: If that person is an Ip you've never seen before or a new user you may want to go check. And this is how they police a lot of what happens on Wikipedia. In fact, if I show this in real time, you you don't even see the the deletions because they are fixed so quickly.
Fernanda Viégas: The last thing I want to show here is chocolate. So this is the
Fernanda Viégas: article on chocolate, and if I visualize it by versions, you see this beautiful zigzag.
Fernanda Viégas: What this is It's a net edit war
Fernanda Viégas: so, and i'll show you what it is back here, someone, This person, Daniel C. Boyer, added. This little white paragraph here that says extremely rarely melted chocolate has been used to make a kind of surrealist sculpture, called
Fernanda Viégas: the White Stripe, survives for a while, and then someone says, Removing boy or invention,
Fernanda Viégas: Daniel Siboria comes back and says, Reverting Collage is not a boy or invention.
Fernanda Viégas: Google search for chocolate. Collage finds only Boyer
Fernanda Viégas: reverting.
Fernanda Viégas: Leave your humbug out reverting, and so forth. So this is really a fight, and it happens on Wikipedia, and it's too bad because Daniel Sibora gets tired and leaves,
Fernanda Viégas: and it's unfortunate because Martin and I did a search for a chocolate collage, and it does exist. But such is life on Wikipedia. The last thing I want to show about this visualization is that we also have a mode where we got rid of all the author colors, and we just colored the text based on how old it was. So the older it is, the darker it is.
Fernanda Viégas: It was really nice to see that there is a lot of old text on Wikipedia, and one of the things that's interesting to us is that we think we're thinking about in this case about old texts as a proxy for quality. Because if you have communities that are, you know where nobody in the community touches the text. Chances are it's high quality, high quality text.
Fernanda Viégas: Ok. So another example of visualizing, visualizing text and visualizing activity around text.
Fernanda Viégas: Now let's turn to something very, very different.
Fernanda Viégas: We're going to go back to numbers, but we're going to go back to numbers in in terms of massively high dimensional spaces, and we're going to jump into the present and talk about machine learning and how visualization can help.
Fernanda Viégas: I only have time to focus on one area today,
Fernanda Viégas: and it's going to be kind of embeddings.
Fernanda Viégas: And so we're going to talk about kind of the hello as a as a warm up. We're going to talk about one of the Hello World data sets of machine learning which is amnest which you can see here. It's nothing more than handwritten digits. Okay, And the idea is to get a system that can
Fernanda Viégas: separate the zeros from the ones from the two's, no
Fernanda Viégas: no matter how bad your handwriting is. And so the way we do this is that we can turn images into vectors,
Fernanda Viégas: and what we do is we literally go pixel by pixel, for each one of these images, and we give it a value. Each pixel has a value based on its color.
Fernanda Viégas: So if the pixel is black, the value is zero. If the pixel is white, the value is one, and if the pixel is somewhere in between,
Fernanda Viégas: the value is somewhere in between. So I do that for literally every pixel I end up with a vector and the vector in this case it's a vector of seven hundred and eighty, four dimensions, because I had seven hundred and eighty four pixels.
Fernanda Viégas: Okay, so far, so good. I do this for literally every digit in my data set,
Fernanda Viégas: and the good news is that now I can work in math right? I've gone from images to math. I can actually just look at how similar these vectors are, and once I have that we can map. Basically I have data that I can map. And the the demo I want to show you now is a visualization we created
Fernanda Viégas: called the embedding projector, and it is available. It's open source. It's it's publicly available and what it does is it tries to visualize. It projects these highly dimensional vectors into a three D space.
Fernanda Viégas: And what I have here is the same mnist. Ah, data set that we were looking at. They are all digits. I know the ground truth. In other words, humans have gone through each one of those digits and said, This is a zero. This is a seven, And this is how I am coloring
Fernanda Viégas: these images that you see here. So the colors are the truth,
Fernanda Viégas: and the clusters are what my system is doing. It's best to try to kind of like put things in where it thinks is the right places. So it's. You know the way I think about it is when we do brainstorming exercises, and everybody has a post-it note. And then afterwards you try to cluster the post-it notes.
Fernanda Viégas: That's what this system is trying to do it's trying to cluster these post-it notes,
Fernanda Viégas: and I can see that it's doing somewhat of a good job sometimes, and sometimes not so much So, for instance, this is a cluster of six, a number, you know, digits. I can click on it, and I can see the nearest neighbors. They happen to be all sixes for for this, which is great. But here it's getting a little confused between five and three. So if I click on an eight,
Fernanda Viégas: if I click on this eight. Look! The nearest neighbors are five and eight and nine, and it's kind of a mess. So this starts to tell me how well or not, my system is being able to cluster things that are similar, and this is really important for me to try to improve my system for me to try to debug my system. And so this is one way in which data visualization can help.
Fernanda Viégas: Okay,
Fernanda Viégas: Now, let's talk about a more real world situation. So when we were at Google, one of the things that was really interesting that happened a number of years ago is this multilingual translation?
Fernanda Viégas: So, Google, As you know, there's Google translate. And Google was interested in starting to use machine learning for translation.
Fernanda Viégas: And one of the things that had always been the case is that you would have to train a specific model for one pair of languages. So you have a model for English to French, and then back and forth.
Fernanda Viégas: You have a different model for English to Portuguese, and so many models depending on the pairs of languages you want to translate.
Fernanda Viégas: At one point people at Google started researchers at Google started experimenting with a single model that maybe they could try to put in multiple languages,
Fernanda Viégas: and they started to get interesting good results. Um! And they were kind of curious about what was the massive internal state? Ah! Of of these languages in the system? How was the system, thinking and separating these languages. So let's talk a little bit about this. One of the things that ah happened on these systems is. Imagine the system had been trained on
Fernanda Viégas: pairs of sentences between English and Japanese,
Fernanda Viégas: Ok. Back and forth,
Fernanda Viégas: English and Korean, back and forth.
Fernanda Viégas: This is all the data the system had ever seen
Fernanda Viégas: without it ever having seen one sentence go from Japanese to Korean,
Fernanda Viégas: it was able to do translations, high-quality translations between Japanese and Korean, and This is what we call zero shot. So it never had training data that looked like that. And yet it was doing a good, a good job which is surprising. And so one of the things that the scientists
Fernanda Viégas: came to us, and we're puzzling about is, what does the embedding space look like for these multilingual systems, and what i'm showing you here. These are just abstract. It's not real data, but it's kind of like the image that the scientists had. So the image on the left.
Fernanda Viégas: It's kind of imagine I am. Each one of these dots is a sentence,
Fernanda Viégas: and they are colored by different languages, so let's say Japanese is green and English is blue, and then Portuguese is yellow.
Fernanda Viégas: What the scientists couldn't figure out is, is the system dividing the languages into different corners, and then mapping for translation between these different corners?
Fernanda Viégas: Or is it more like what you see on the right, where the system is bringing together multiple languages,
Fernanda Viégas: and, despite say strings looking very different, like I have a string that says home and another string that says Kaza,
Fernanda Viégas: does it know that the semantic meaning is the same. Does it not care about the strings right? The fact that it comes from different languages? So this was the question that they couldn't answer. And so what we did is, we decided. Well, let's try to visualize this, this, this space of embeddings and let's see what it looks like.
Fernanda Viégas: So imagine. Because again we're dealing with sentences. Imagine a sentence like the stratosphere extends from ten kilometers to fifty kilometers, an outage altitude,
Fernanda Viégas: and what i'm going to do to and I up to an approximation. I'm going to create a little dot for each one of these words, and it may look something like this again. This is all not real data, but just to give you a sense. And so the dots for the sentence will be scattered somehow, but things like ten and fifty, because they're both numbers will be closer together, because there's a certain similarity between them. And then what I do is, I connect these dots
Fernanda Viégas: because that is the sequence of my sentence. Okay. So I visualize now one sentence in the embedding space. When I translate that sentence to a different language.
Fernanda Viégas: Does it look like this? Where English is red and Portuguese is blue, and they are separated?
Fernanda Viégas: Or does it look like this? Where I have English and Portuguese roughly together in a cluster.
Fernanda Viégas: Okay, and this is what we try to to answer. So now let's take a look
Fernanda Viégas: at the actual visualization. So this is a visualization of a system that takes three languages, English, Japanese, and Korean. I am visualizing sentences, and I am visualizing them in different colors, depending on their source language. So if the source language is is English, it's going to be blue. If it's, you know Japanese, it's going to be yellow and so forth. Okay,
Fernanda Viégas: complicated image.
Fernanda Viégas: I don't know exactly what's going on. But the main point here is, Remember those two pictures I showed you.
Fernanda Viégas: Do the colors separate into different corners, so do I have a corner of only English, only blue, and another corner of only Japanese only red?
Fernanda Viégas: Or are these colors coming together and clustering. They are coming together in clustering. I don't have different neighborhoods here of only a single color, and so let's look at one at our example sentence that
Fernanda Viégas: we were looking at with the stratosphere. So I just highlighted. This tratosphere is in the range of ten kilometers to fifty kilometers,
Fernanda Viégas: and my nearest neighbors, regardless of the language, are all in that little cluster together in the same cluster. So this was super interesting to us, because it was the first time we were seeing
Fernanda Viégas: what looked like the the the initial signs of an interlingual of a universal language, the system was able to actually bring together different languages based on the semantic meaning of these sentences. And so that was very exciting. And it was a real like, you know, scientific insight, for for the team who had not been able to resolve this question of of what the space looks like,
Fernanda Viégas: and you may be thinking Well, that's great. But why does that matter?
Fernanda Viégas: Well, let's go see a sister. Ah, system. So this is the same kind of visualization, but of a system that takes Portuguese, Spanish, and English. And if you look at this
Fernanda Viégas: you will see a neighborhood of just red to the side.
Fernanda Viégas: It looks very different. We looked at this. We're like what is going on here. These are the same kinds of systems. What's going on? So we downloaded all the data, and we ran a statistical analysis of the quality of the data of the translations, and we found that, sure enough for this system, the quality was low.
Fernanda Viégas: And so, in other words, if your system cannot solve the geometry of the space matters here. If your system is not being able to bring these languages together in terms of their semantic meaning,
Fernanda Viégas: you have a problem. Your Your system needs to be debugged, improved. You maybe need more training data. But this is a problem. And so this is another way in which visualization can help us kind of get a little bit inside the inner workings of of these systems.
Fernanda Viégas: Okay,
Fernanda Viégas: Now, everything I was showing you here, these projections. Here they use a protection technique called T snee, which is a non-linear.
Fernanda Viégas: So again, remember, I have a massive amount of dimensions. And now I am projecting all of those dimensions into unknownd. So I can try to understand what's happening.
Fernanda Viégas: This comes at a price. So these visualizations are highly used in machine learning.
Fernanda Viégas: They are tricky. And let's talk a little bit about this.
Fernanda Viégas: Things can be deceptive in these visualizations, because Um and i'll show you a couple of very simple examples here. So imagine I have an original synthetic. I'm: I'm. Showing you some synthetic data here. Okay, with three clusters. Okay, So I know this is the ground truth. Okay? And then I do a T snee again, the same projection onto this data set.
Fernanda Viégas: And it looks like this.
Fernanda Viégas: So I have all of a sudden lost this global relationship between my clusters, because now they all look like they are equidistant, which they are not. And this is unknown problem with t snee, even though it's very useful that it does a good job of keeping the local clusters. So you can still see there are three clusters. It's not lying about that, but it doesn't do a good job of keeping the global structure
Fernanda Viégas: of the original space high dimensional space. Okay? And There are other things, too. There are. T. Snee has a number of parameters that we can play with. So one of them is called perplexity. And again, if you look on the left. This is my original data set. This is the ground truth,
Fernanda Viégas: and depending on my perplexity that I may choose. These clusters look very different, and so which one is right, which one is the truth. And so
Fernanda Viégas: Martin and I wrote a a whole article about how to try to use these in in better ways, but these are used across the board, and in a sense they are kind of the new scatter plot
Fernanda Viégas: of our age in terms of machine learning, and I think we have a long way to go still to try to understand. How do we? How do we work with the distortions that they create.
Fernanda Viégas: So let's just pause there for a moment, and think like, Is this useful at all? Or are we lying to ourselves as we use this visualization technique, right?
Fernanda Viégas: The truth is, we are already in a world where we use distortion all the time. Think about it
Fernanda Viégas: any time we have a map of the earth, the
Fernanda Viégas: we are destroying
Fernanda Viégas: the data right? Because basically it's impossible to show something from threed to twod without distortion.
Fernanda Viégas: But at this point we know enough about the distortions. We know enough about each projection technique. Hopefully as we're looking at these maps that we know how to take those distortions into account. So, as
Fernanda Viégas: again because I grew up in Brazil,
Fernanda Viégas: I
Fernanda Viégas: would hate the way they would show the world map, because one of the most familiar and most popular projections is the Mercado Mercator Projection, and it makes Brazil tiny, and it makes everything in the in the northern hemisphere. Brazil and the rest of the southern hemisphere, tiny and the northern hemisphere huge, and i'm like No, Brazil is just as big as the Us. But you would never know if you didn't know it was destroyed right.
Fernanda Viégas: All I'm saying is,
Fernanda Viégas: as long as we understand some of these distortions, and we work critically with them. It is incredibly useful still to map this kind of high dimensional data.
Fernanda Viégas: Another way in which data visualization can be helpful in terms of machine learning is to help people pro these systems. And here i'm talking about non experts, non-machine learning scientists. So let's talk about one example.
Fernanda Viégas: Everybody talks about fairness and bias in in the machine learning in Ai systems. And one of the things that I think is really important as we have these conversations is that
Fernanda Viégas: I Don't think these decisions should be left only for engineers and and machine learning experts to make. As some of these systems, you know, hit real communities and societies. So how can we share the responsibility?
Fernanda Viégas: And I think part of it is that people need to understand what some of the trade-offs are
Fernanda Viégas: these systems? And so how do you do that?
Fernanda Viégas: I think visualization and simulation can be real, powerful tools in that effort. So i'll show you one little example that Martin and I did
Fernanda Viégas: um. We're going to play a little game. We're going to pretend we're a bank, and we're going to decide who to give loans to each person who comes to our bank looks like a little circle, and they have a credits core. My credit score is a make-believe credit score. It goes from zero to one hundred
Fernanda Viégas: higher is better. Um! Anyone colored in the light blue color would default on our loan. Anyone in a dark color would pay us back. So we definitely want to give loans to these people to the dark-colored people.
Fernanda Viégas: Okay,
Fernanda Viégas: everybody comes to our to our bank. We set a back bank threshold above that threshold we give loans to below. We deny loans,
Fernanda Viégas: we're all happy except that real life isn't like this right kind of no matter where I put my threshold. I'm going to make incorrect guesses. I'm going to give loans to people who don't pay me back, and I'm going to deny loans to people who would pay me back.
Fernanda Viégas: Okay,
Fernanda Viégas: this is what this is still kind of made up, but it's what a distribution in real world starts to look like, and you can see that
Fernanda Viégas: ah kind of no matter where I put my threshold. I have a mix of people who pay me back and don't pay me back.
Fernanda Viégas: Okay.
Fernanda Viégas: Now, Um.
Fernanda Viégas: Together with these things, I have indicators. I'm a bank. I care about profit. I care about hopefully about How many people am I giving loans to? So I have things like, you know, positive rate.
Fernanda Viégas: Um! Of all the people who come to my bank, what is the percentage of people who are getting loans. In this case it's fifty, two percent. Okay, Ah, percentage of correct guesses, incorrect guesses, and so forth. True, positive rate
Fernanda Viégas: of all the people who come to my bank and would pay me back. What is the percentage of people i'm giving loans to so good news for me. I'm giving loans to most of them eighty, six percent.
Fernanda Viégas: Okay, Now let's think about something broader than this. I have two populations that come to my bank, and they have different distributions, and I know that.
Fernanda Viégas: How can I be fair to them? How can I be fair in deciding who who gets alone. So we created this visualization so that people could play with it online. So i'll show you what this looks like right now,
Fernanda Viégas: all right. So This is the visualization, and I'm going to make it a little bit smaller, so that it fits here.
Fernanda Viégas: So I have my distributions. I have the blue folks, blue people, the orange people who come to my to my bank. You can see that I can change threshold if I don't care at all about about fairness, and all I want to do is maximize my profit. I have a preset up here. I'm going to click on it. Yep, this is my maximum profit. Give it the given distribution of people.
Fernanda Viégas: Okay.
Fernanda Viégas: But if I decide to be fair,
Fernanda Viégas: how can I be fair? Maybe I say, you know
Fernanda Viégas: I don't care if you're blue. If you're orange, you come to my bank. I treat you the same. So this is called group unaware. I optimize, for I have the same threshold for everybody. I treat everybody the same. That's what fairness means to me.
Fernanda Viégas: Okay, Well, once you do that, then let's look at your positive rates here.
Fernanda Viégas: So of the blue people coming to my bank, I am giving loans fifty, two percent of the time of the Orange people coming to my bank. I'm giving loans. Only thirty percent of the time. Is that fair?
Fernanda Viégas: Maybe Maybe not. So let's try harder. Maybe we can do something called demographic parity. So i'm going to click on this button next. And now i'm optimizing for the positive rate. So now I have the same percentage of blue people and orange people receiving loans from my bank. What happened, though, is that I had to set different thresholds.
Fernanda Viégas: Maybe that's okay. Maybe that's what fairness means to me.
Fernanda Viégas: Now look at true positive rate, though the people who would pay me back,
Fernanda Viégas: I'm only giving it so i'm giving seventy one percent to the of loans to the
Fernanda Viégas: you know, good orange people, but only sixty, four percent of the time. Am I giving loans to the good blue people? Okay, Is that fair, so there is something called equal opportunity. So I can click on that. And now i'm optimizing for that parameter,
Fernanda Viégas: now
Fernanda Viégas: long story short,
Fernanda Viégas: the whole point of creating this visualization, and we created it. We put it online. It was an an accompaniment to an academic publication that had all the math. So the the paper had all the math, and we decided to visualize it.
Fernanda Viégas: It was to start to bring a broader set of stakeholders to this discussion around fairness. So. Um! And what we saw once we put this online is that people resonated with this and that they wanted They started having conversations on Twitter and be like. Oh, I always thought demographic parity was the way to go. But now I can see that you
Fernanda Viégas: and make no mistake. At this point there are hundreds of different metrics for fairness,
Fernanda Viégas: and part of the point here is that
Fernanda Viégas: none of them are perfect. There is no perfect
Fernanda Viégas: all of them are a set of trade-offs. And so, once you were choosing whatever metrics make sense to you understand that you were always going to be making a set of trade-offs. And so one of the things that was really nice of doing this work was that we started getting emails. So for instance, we got an email from
Fernanda Viégas: a Criminal Justice Department in one of the States here in the Us. And they were asking us, you know. Obviously, we're not a bank. We're not giving loans. But we are using algorithms, and we are making trade-offs in our choices here. Do you think a visualization like this could help people in the Department understand better the choices they're making, and we're like, Yes, we definitely think so. So, in fact, we created a whole tool at Google that is open source and
Fernanda Viégas: publicly available, so that you can actually do these kinds of visualizations and change thresholds and understand what are the choices you're making? How do those choices impact the different people, or it may not be people with maybe products, or whatever that you're dealing with,
Fernanda Viégas: so broadening the sense of,
Fernanda Viégas: you know, broadening the number of people who can think critically about these mathematical trade-offs sometimes can be a really empowering.
Fernanda Viégas: I want to talk very briefly about different perspectives and uses of database today, because this is interesting.
Fernanda Viégas: There has been, especially in Academia. A lot of attention has been given to things like um. Ah, Edward Tufte, so i'd say the the The book you see here on the left is by Edward Tufti. He's one of like the Popes of of Data visualization, and has done amazing work to kind of educate people about how to make
Fernanda Viégas: useful, effective data visualization.
Fernanda Viégas: He also has a very specific take on things like your visualization should be neutral care about data, ink, ratio
Fernanda Viégas: just by taking this notion just might make, you know, bringing up this notion of a neutral point of view,
Fernanda Viégas: he tends to overlook a whole lot of choices that people make when they create visualization. So in contrast to that today there is a whole set of different voices that are talking about data visualization. So what you see here, data feminism on on the right,
Fernanda Viégas: where people are saying
Fernanda Viégas: nothing, we do with data. Initialization is neutral. There is no neutral. Even when you decide to collect data, let alone visualize. You are making a series of choices, and it's important to acknowledge those things. So there is a whole discussion ongoing discussion about
Fernanda Viégas: the importance of emotion. How do we study emotion in relation to data visualization and the responses that people have, and I think the reason why I bring this up right now. I don't have a lot of time to talk about this. But
Fernanda Viégas: um! It's because I think, even in a in an era where we are faced with so much misinformation, it is really important for us to think about data visualization as a rhetorical device It is. It is a common, it's a, you know, communication device, and like any other communication device, you can manipulate it. You can use it in whichever way. And I think it's really important that we realize that there are different points of views that people are using when they create
Fernanda Viégas: data visualization, because i'm running out of time. I'm going to skip very quickly.
Fernanda Viégas: And i'm going to talk. I'm going to finish off with Ah! Data visualization as art. So many years ago Martin and I were confronted with a question. The Boston Magazine came to us and said, Can you visualize Boston, and this is a print magazine. It's not interactive. We're like. Oh, my gosh! How do you visualize a city?
Fernanda Viégas: So we started thinking about what's unique for us about Boston, and one of the things that we thought about is the seasonality. The seasons are very well marked here, and we're like. Is there any way we can capture them.
Fernanda Viégas: So we decided to look at a public collection of user-generated images of the Boston Common The Boston Common is the biggest Public Garden in Boston. It's kind of like the central part of Boston, and
Fernanda Viégas: there was something called Flickr back. Then, when we were doing this project, and people would post public photos, and they would label them. And a lot of those photos were licensed under creative comments. And so what we decided to do is, we decided to download a whole year's work of pictures
Fernanda Viégas: of the bus in common, and then we bend those pictures by all the pictures from January, all the pictures from February, March, and so forth.
Fernanda Viégas: Okay, that was our data.
Fernanda Viégas: Then we did the following: We decided to kind of squeeze the color from those pictures,
Fernanda Viégas: and with this we started counting colored pixels. That's what we did, and so imagine you are. You have a color cube. You have a color space what i'm showing you here, and you were literally using very pure hues right for each one of the little color cubes there, and you just fill it up. You just make each cube bigger. The more pixels of that color and some of the other colors are going to be smaller.
Fernanda Viégas: And then what we decided to do was to remember we had a bin per month,
Fernanda Viégas: and so we decided to connect those months in terms of color. Imagine a ribbon of color that gets thicker,
Fernanda Viégas: depending on the month of the year. And so this is what the visualization looks like.
Fernanda Viégas: So this is a whole year's worth of colors in the Boston Common,
Fernanda Viégas: and the year starts at the very bottom with January at the bottom, and you can see just the amount of race that exists there. That's just That's that's our winter. Lots of lots of snow here. Um right and then, but still lots of blue sky, too blue never never leaves us, which i'm very, very grateful for. But then, as you and it moves cornwise right so if you keep going up to the to the left. You see you start seeing
Fernanda Viégas: like a thin line of like fuchsia or this magenta.
Fernanda Viégas: Those are the these are the flowers starting to grow in the park during spring, and then you have a bulge of green around June, and those are the the, you know the leaves coming out. If you go all the way to the bottom, right like October and November. That's when you start to see the earth tones of of the famous fall foliage here in Boston, And so you can really see how these things are changing
Fernanda Viégas: kind of ebb and flowing over time. And the reason why we decided that this should be a circular display is also because time and seasons are cyclical. So to us it made a lot of sense to go with a circular display. Here
Fernanda Viégas: I want to finish with another
Fernanda Viégas: art project that that Martin and I did, based on data. And this was we were interested in visualizing the wind, and this was again during a very cold winter here, so who knows what's going to happen this winter? Maybe it's not so, Code. We won't come up with a visualization. But we were looking at the when we wanted to understand. Is there anything we can do with when we that we could visualize the wind? And
Fernanda Viégas: truth is,
Fernanda Viégas: wind has been visualized, in fact, for hundreds of years. What you're seeing here are two screenshots of vector fields where you aggregate. Then you use these arrows to show both direction and the length of the arrow, to show the strength of the wind. Um! And these work fine. But we wanted to make sure that there was. You know that we could do um.
Fernanda Viégas: We could do something more um dynamic. We could use the web, and it's dynamic form, and also because the wind is dynamic. So we wanted movement. And so what ended up happening? I will show you now, if I can go there,
Fernanda Viégas: let me. It's the wind map. So this is the windmap.
Fernanda Viégas: It's it's a public website. It's live, and it shows the wind.
Fernanda Viégas: It actually it's stuck right now in the pest a little bit. I'm two hours back. This was the win two hours ago. Um in the Us. And what you can do is you can start to zoom in,
Fernanda Viégas: and I can see that Boston, where I am right now, has a lot of wind coming
Fernanda Viégas: kind of from the um southeast, which is good news to us, because it means it's going to be warmer wind this time of the year than ah otherwise. It also has really broad patterns like this. Is all this wind going towards the east at the top of the country, and it has very delicate patterns here
Fernanda Viégas: because of the mountains. So this if I zoom in, If I start to zoom in around Denver, Colorado, you can start to see how the wind kind of parts and and comes together because of of
Fernanda Viégas: because of the terrain.
Fernanda Viégas: Um, Another thing we started seeing when we created this visualization was just how much the wind changes day after day. So, for instance, there are days like these, and I'm going to bring up a day different day, when you know Canada steals all of our air, and not even the mountains can keep us from keeping. You know the air here. Um! There are days like this,
Fernanda Viégas: and this was the first time after we created this visualization that we were seeing a hurricane make landfall in the Us. This is hurricane isaac in two thousand and twelve. And
Fernanda Viégas: again, because This is a real-time visualization. We started getting emails from people. So I remember one email we got was from someone in New Orleans saying, i'm here in New Orleans. I'm hunkering down praying that this thing
Fernanda Viégas: it passes over. I'm looking at your visualization, And it was a really,
Fernanda Viégas: you know, powerful thing to get that email at that moment as these as this was happening. Um, in fact, uh later on. Um, this uh was Hurricane Sandy making landfall, and we were kind for for a change. Boston was kind of in the past. It was not, you know, the I of the hurricane, or anything like that. But we were also looking at the visualization and hoping that things would be okay.
Um. And so
Fernanda Viégas: it it.
Fernanda Viégas: Visualizing data in real time
Fernanda Viégas: has this added effect and power that we had not seen before. In fact, we thought oh, so just one note. This is the same data that you see here.
Fernanda Viégas: So um! It's interesting to me just how much the different technique can change what you, what you see in the data and the fact that there is a lot more aggregation happening in these vector fields than there is here. And yet our eyes can resolve these images so easily, I think, is an unintuitive but interesting point.
Fernanda Viégas: But the other thing about the windm is just the response it started to get. We put it up as an art piece. If you think about it, we're not labeling much anything. We don't show state boundaries. We don't show capitals. We don't show many things. We don't even use color,
Fernanda Viégas: which is one of the major dimensions you have in data visualization, just because all we wanted was to see the shape of the wind. And so we thought about this as very much an art piece,
Fernanda Viégas: and not not utilitarian in any way. And yet the response was very interesting. So farmers started using the map to decide when to spray their props, for instance, scientists, and would look at the map to better understand bird migration, butterfly migration teachers would sit with the kids and and look at the wind map and try to figure out weather forecasts.
Fernanda Viégas: Um pilots. We started getting emails from pilots, both commercial pilots and military pilots, saying that they were using the windmill, at which point we're like.
Fernanda Viégas: Oh, no, no, no, no, no, please, no! This visualization uses surface wind, which is from zero to twelve meters.
Fernanda Viégas: It's definitely not where you fly an airplane, and the winds up there are very different, and the pilots know this, and we had a very clear disclaimer, saying, This is surface wind.
Fernanda Viégas: And yet we kept getting emails and emails and me to the point where we decided to put a disclaimer on our website, we said,
Fernanda Viégas: Please do not use the map or its data to fly a plane, sail a boat or fight when wildfires
Fernanda Viégas: because we felt responsible. We kept getting emails.
Fernanda Viégas: And then, after we put the disclaimer, we started getting emails like this, where they're like, Yeah, Yeah, I see your disclaimer. But please respect the power of this visualization in promoting the prevention of wildfires. So
Fernanda Viégas: it was interesting, and it told us a couple of things which again, when you make. I think this amount of data
Fernanda Viégas: very easily understandable to people, because again, this is data that that is already out there. It's government data which i'm very grateful for um. It just takes a life on its own, and people appropriate it and use in whichever ways they want to use, which is also really
Fernanda Viégas: interesting and powerful to see. In fact, the technique ended up influencing
Fernanda Viégas: most media outlets. Ah! In the way they show whether and when specifically, we did not obfuscate our code on purpose. And today, when you see, you know maps of wind in all these different Ah, ah! Places! It actually comes! Ah! From the wind map, which is really cool really, and these are professionals. We're
Fernanda Viégas: doing something for art. And yet, you know, it ended up there. It's also part of the permanent collection at Moma, and it's actually being shown right now in New York City. So I just want to end on this note to say that hopefully,
Fernanda Viégas: you know, after all these Demos Um, I've been able to impress upon you that data visualization is much more than visualizing numbers, only it's not just for individuals. It is definitely not just for experts. Um. And I I personally think about this as a very broad and expressive medium that invites it.
Fernanda Viégas: Big variety of stakeholders into the world of data. And so with that, Thank you so much. I would love to take questions. If there are any,
Michael Littman: you can't hear everyone clapping, but we peaked at about two hundred and five people out there listening to what you had to say. And fantastic job. Thank you for sharing.
Michael Littman: Ah, this the the story of of the work that you do, the the the actual visuals that are are high by popping. This is great. Yeah, we do. Have. We have a bunch of questions that that i'd be happy to share with you as we go through. So i'm going to do them in
Michael Littman: chronological order. So we're going to be harkening back to the beginning of the talk
Michael Littman: You were talking about the difference visualizing the search term completions.
Michael Littman: Yes, what are they? They They call it the the Google suggest.
Michael Littman: Yeah, but but the there's a there's a group, is it? Is it wired that does this. They actually turn these into interviews for people they ask, they say, what would Fernando have to say about? And then, you know, or is Fernanda really blank, and then and then they'd actually ask the person the questions that they Google has. Um. Have you tried more than two search terms? You had this this beautiful way of visualizing the Venn diagram of two. What happens when you do more than two, does it? Just?
Michael Littman: Oh, that's that's interesting. We have not tried. That, too, seemed like the obvious thing for us to do. Venn Diagrams are hard. They're hard. So once you get out of two or three. It's
Fernanda Viégas: It's a good. It's a good question. I don't have an answer. No, we have not tried that.
Michael Littman: Okay, All right, fantastic. And I was noticing that some of the arrows Crick crossed, and I didn't understand why they were doing that. They're kind of changing the order that the arrows were coming out to where they landed.
Michael Littman: Oh, was that doing that? Let me see. Yeah, yeah,
Michael Littman: that's interesting. So one of the things we try to do, and this could be Why, um. So there is an order in which Google the Api gives you the most popular to the least popular, if you will. So there's an orator there
Fernanda Viégas: for both of those terms. Right that you're quering now, because we want to emphasize the places where the two queries have the same completion. It could be that
Michael Littman: the order gets messed up, and we're going to have to. You know the the the arrows may end up. It's been something to the outside when it was actually more central before. But
Michael Littman: yeah, I get that. And I saw that. But it was also before you put in the second query. One time there was an end.
Michael Littman: Oh, it was getting this, Really,
Michael Littman: I didn't know what. Okay? All right. So maybe that was either. Maybe it had a guess as to what your second query was going to be, and it was pre-loaded.
Michael Littman: Oh, oh, oh, no, no, no, I'm looking i'm. Looking. Okay. There is one thing that happens with text. That is one of the challenges of text, which is,
Fernanda Viégas: imagine I have an ordered list, which I do in this case.
Michael Littman: It also matters the length of the text. So if I have, let's say, will Brazil win the World Cup? That's a long string, or will Brazil, and whatever it is, Will Brazil nuts hurt me. But if I have a very short completion that happens to be very popular,
Michael Littman: that arrow is going to be longer than the other ones, and it's going to do so. It looks like they cross, but it's because literally we have different lengths of strings. Oh, wow! And and we are
Michael Littman: right. Yeah,
Michael Littman: we are censoring the text.
Michael Littman: Yeah, thanks for clarifying, all right. So And that question was from anargia dos. So i'm gonna.
Michael Littman: I guess that was answered, Live all right. So, um you had said in the Wikipedia thing was very cool. You had said that the older text was black, but we were thinking. Maybe the older text was white.
Fernanda Viégas: No, the older text um, the the the older the text, the darker it becomes,
Michael Littman: and so new additions will be white, The bright color. Yeah,
Michael Littman: yeah, does that. Is that a confusing thing with respect to visualization? Because when there was missing text that was also plaque.
Fernanda Viégas: Oh, yes, that's a good point. Yeah, so it's It's
Fernanda Viégas: It's the kind of thing. Let me put it this way. We built that visualization, so that Martin and I,
Fernanda Viégas: as curious scientists could understand those editing dynamics. And so we knew exactly how to read those colors. But to your point, and that's a very good point. It's a very important point. If we had released this as a tool
Michael Littman: for anyone, we would definitely have to have a very clear caller legend and make it and and annotate things, so that so that there would be no confusion on how we're using color. So yeah, it very, very good point about. Yeah, we were doing. We were creating this little microscope for ourselves. But yes, when you release, you have to really pay attention to annotations and legends. Okay, So that observation was from the Glen Langston.
Michael Littman: Uh, let's see. So um
Michael Littman: I don't
Michael Littman: just so that everybody knows we have this slot until one thousand two hundred and thirty Eastern time. So that's another twenty minutes. So i'm going to prioritize a little bit with the questions. Let's see um
Michael Littman: all right. So uh, so in Drakeiki,
Michael Littman: who, I think, one of my program managers said the problem with Tee's knee. So he he recognized that you were using. Excuse me before you had said, it is that a small change in one of the parameters could yield vastly different visualizations.
Michael Littman: Yeah. And look at this data. It's easy to fiddle with the parameters, and the results make sense. However, data is abstract. One could not know what the clusterings have clustering to, and this relates to something that you actually said in the talk as well, which is sure distortions happen.
Michael Littman: But as long as you know what the distortions are, and you can adjust for them. That's fine. But, as we know, with the Mercator projection,
Michael Littman: Don't think most people know what that that there is a distortion, and I know that there's a distortion, but I actually don't remember what it is. It's something about
Michael Littman: kind of where the where you are on the the thickness of the anyway. So I don't. I don't know how to do that. But the Disney I also. I've used it, but I I don't know what it's trying to optimize. What is it trying to capture? So I can't counter the distortion at all. What What do you say to that? Is there a way to use visualization to understand how to understand the visualization.
Fernanda Viégas: So this is one of the things that I think should be a very active area of research.
Fernanda Viégas: My sense. My good feeling is that one of the things we could start doing right now is a couple of things. So one is
Fernanda Viégas: Let's say you find it's to combine data, visualization and statistics. Right? So
Michael Littman: let's say you were interested in it. Of course those are both things that are hard for people, probably more. So the statistics.
Fernanda Viégas: Yeah. Yeah. So yes, let's okay. So let's take take a step back
Fernanda Viégas: of all the visualizations I showed today.
Fernanda Viégas: The Tsni kind of visualization to me is the least accessible to regular people. Right? It is the most. It is the kind of visualization that you're going to have your experts.
Fernanda Viégas: Look at
Fernanda Viégas: having said that. I also believe we're now in a world where you're going to have cross-functional teams. For instance. Let's say you are trying to launch a product or you are trying to decide whether your machine learning system is doing a good job.
Fernanda Viégas: Chances are you're gonna have a situation where you're uh software engineers who know the system will be looking at T snee. Um. You may have a research scientist who knows the data hopefully understands how to interpret tea, sneaky pots. Look at tea sneaks. But then you may want to show some of that to your project, manager, or even to your executive, and I think that's where a layer of translation, and also
Fernanda Viégas: maybe some checks with statistics are really important. So you can do things like,
Michael Littman: okay. We see this cluster. How reopens this cluster? Can we do things like? What is the centroid? How how much, how pet, how cohesive is this cluster when we look at all the the metadata about those data points. Do we see things that to us look like a cohesive cluster? So I think more analysis is important. Want to make sure you are not projecting things that aren't there, and too, so that you can have
Fernanda Viégas: a real critical conversation with other stakeholders that are not going to be as versed as you on how these plots are distorted.
Michael Littman: Yeah. And so this is. I have found this to be really relevant in my still somewhat new job as as as a division director. So um on Julie Bums bombs, I from, I think, from Nsf. Said I see an analogy. So this is now looking at your your bank loan. Visualizations
Michael Littman: analogy between this example of bank loans and our peer review process in terms of fairness inherent bias. This is something that we think about a lot. And and Nsf: we really want to try to do the right thing. But we've got lots of data. We've got lots of sort of confusing signals that we're all trying to integrate together. We've got lots of
Michael Littman: stakeholders that are pulling us in different directions, and I know since that since I've arrived, people have presented me with visualizations of for example, these are the proposals that are coming in, and
Michael Littman: I don't know if i'm dumb, or if i'm too sophisticated in my use of visualizations. But I look at them, and I say I need to know how these were generated because I don't know how to interpret these blobs of color.
Michael Littman: Yes, yes, and it doesn't seem to be. It's becoming common practice for data visualization to be accessible, because i'm being bombarded with it. But it doesn't seem to be common practice yet to say, Oh, and by the way, here's what Here's what It means right. It's just it's just It's just hitting us in the eyes, and we're reacting to it. But I don't know,
Michael Littman: reacting appropriately. So do you have like best practices. I know in your artwork. Maybe it's less significant. But when we're actually trying to make decisions based on this data, how how should we be informed about how it was generated. In the first place.
Fernanda Viégas: Yeah. So I think to your point. There is
Fernanda Viégas: a big difference between like traditional data visualization bar charts like those. I think we know we are on solid ground.
Michael Littman: It's a very literal in a sense, right
Michael Littman: interpretation Also,
Fernanda Viégas: like I feel like we've come a long way about things like label, Label your axes, label your units right? Have a title. Explain to me, where is this guy? So
Fernanda Viégas: all of those practices should should somehow translate. But when you get to a place like t snee or u-map, or those things axes don't mean anything right, and So
Fernanda Viégas: it needs to be clear to to data scientists that if the axes don't mean anything, you're gonna you better have more um justification, or you better have more data around again, whether or not the the clusters or the relationships you're looking at really stand any test or any statistical test. I feel like those are the things that should be brought to front, even,
Fernanda Viégas: you know, as as you're presenting these things to the people who didn't do the data analysis that you did right so to your point. Like you were the director, you should. It shouldn't just be a screenshot and say, this is the cluster
Michael Littman: That's the starting point. That's the starting point. Only,
Michael Littman: yeah, And so we should be able to poke at it a little bit and see if it's bridal right? Because exactly it's very pretty, and it's evocative to get real. So I like I like that that you are suggesting that additional statistics should be available. It should just be the the dead snapshot
Michael Littman: that's that's great.
Michael Littman: Ah! Somewhat related to this is Andy Rubin asks. So this is in the context of the the overlapping distributions that you were showing and playing with the thresholds for the different subgroups, the the blue group and the and the orange group. Um! That was Ah, it's just a beautiful exercise. I feel like I feel like the New York Times has been doing things like this sometimes, letting people play with it. They should do. Yours. That was like that was really.
Michael Littman: It really does It makes it very vivid that it's trade-offs it's just trade-offs. It's not like
Michael Littman: fair versus unfair. It's like Well, it's fair in this way, and this is unfair in this other way. What matters to you so? But if we were talking about real data, like real bank loan data or real proposal data at Nsf.
Michael Littman: Isn't. It true that that in the real world we don't know the actual distributions in particular. The missing piece is the counter-factual. If we,
Michael Littman: I guess the one the one critical counterfactual is. If we had given this person the loan, would they have paid us back? So the negative people we don't get to find out the positive people we do, and they can be labeled appropriately. But the negative people like,
Michael Littman: and so we don't get to directly visualize that. So we
Fernanda Viégas: we don't have the gods-eye view of the whole thing right? It's true,
Fernanda Viégas: but even so, I think one so the tool that I didn't the work that you saw with the bank loan.
Fernanda Viégas: It was a very small simulation visualization, and then from there, and from the reaction we got, we built an actual tool called the What if tool
Fernanda Viégas: that again is available? It's online it's it's open source where you can visualize your data and to try to understand what your system is doing to your data. So to your point, one of the things. Let's say you're talking about proposals Right
Fernanda Viégas: You could. The idea would be that in this tool you would be able to play with different thresholds. But then you would be able to do things like. Now let me fat. So, instead of just having one threshold, I want to facet these proposals by
Fernanda Viégas: whatever institutions, or by the gender of the pi, or by whatever. And now I want to see what happens if I play with these thresholds in a more sophisticated way. Right so? Yes, to your point. You don't have all the counterfactuals all the time.
Fernanda Viégas: But I think one of the things that we are really missing
Fernanda Viégas: is to give people
Fernanda Viégas: good ways of developing an intuition
Fernanda Viégas: for how these choices affect things in the real world, and I think that's a little bit of what
Fernanda Viégas: ah visualization tools and kind of can get you closer to It's it's developing. This does not look right to me. I don't know, you know, like. I don't know what the ideal is, or if there is a perfect solution here, probably chances are no,
Fernanda Viégas: but I feel like we don't even have the intuition right now,
Michael Littman: right that sometimes the once you develop that intuition, at least, if something is off just really off, it hits you in the in between the eyes, and you can act on it,
Fernanda Viégas: or you can come out of the session and say, Oh, my gosh! The one thing we should be asking for in these proposals is this other piece of information?
Fernanda Viégas: And and we're missing that
Michael Littman: So for the next round can we have an additional piece of information, right? So it's kind of like shining light on on these different dimensions of your data That might be useful. That's great,
Michael Littman: all right. So the the Boston color Visualization got people very excited, so maybe we could do this as a quick call and response. So very best answers to a couple of these questions. Um, Have Jonathan Madison asked. Is there other cities that you have done this with?
Fernanda Viégas: Oh, yes, we did, Rio, where I grew up, and we did. We did a couple, but yes, we haven't shown them to anyone.
Michael Littman: Okay, I think I think there's There's some hunger out there. Okay, Good
Michael Littman: um. Catalina Arkansas I'm. I'm asked.
Michael Littman: But making the loop made a lot of sense because of global climate change. The years are actually changing over time. Would there be any interest in extending that and using time as the X axis and seeing, can we actually see the vibrations changing or the shifts shifting?
Fernanda Viégas: Oh, I would love that that is such a good idea, one of the challenges there
Michael Littman: that we're for this project. We were relying on literally a public collection of photos, so the signal is noisy.
Michael Littman: But yeah, but I really like that idea.
Michael Littman: It's you have biased in all kinds of ways like people are going to be more likely to take pictures of the grass and and just kind of wrapping that up? Sandra Handy asked. Okay, this is great. You You pointed us to a bunch of public examples are the yeah Urls collected someplace like, Can we share them with?
Michael Littman: Oh, yes. So if you just go to my website here, can I i'll put it here.
Fernanda Viégas: It's
Michael Littman: so. Blaine might have to copy it to everybody else.
Fernanda Viégas: Fernando Viegas dot com.
Fernanda Viégas: You can see all of this, all of this work there,
Michael Littman: all right. So it's Fernando Vegas calm. No space, no accent. Yes. Calm.
Michael Littman: That's it all right. And Blaine copied it for everybody. All right. Fantastic.
Michael Littman: Yeah, I I just think I mean It's just wonderful how I want to say tactile. You've made the visuals, which I guess is a kind of sense crossing, but it becomes a thing when you're manipulating. It makes it so much more embody and and relatable. It's just It's just great. And so I know people are going to want to play.
Michael Littman: Oh, good. Good.
Fernanda Viégas: Yeah. The web here, for instance, you can spend hours there. It's It's a time sink.
Michael Littman: Yeah, So the the okay, the wind visualization.
Michael Littman: First of all, what? So this is another example of like me as a viewer, not really understanding what it is that you were showing. So how does the wind readings which I guess, are made in like little spinning cups? That's the big places. How does that get turned into lines that are that are shown across an entire region.
Fernanda Viégas: Yeah, it's a great question. So it turns out the Us. Government relies on a huge network of sensors,
Fernanda Viégas: and it's even more interesting than that. Each State has its own network of sensors, and they all send these readings Ah! To ah to be collected by Noah. And so Noah makes this Ah, this data set available. It's also a forecast data set in the following sense: They know what tries to forecast the wind every three hours, and then we show that forecast. And so,
Fernanda Viégas: the closer you are to that three-hour mark the closer you are going to be to a new reading from Noah. Right
Fernanda Viégas: and literally all it is. It's the direction of the wind. It's the force. It's the speed of the wound. There are other dimensions, too, but we chose just to to literally use speed and direction.
Fernanda Viégas: Um and all. We're doing. It's like the way we visualize this. It's like a particle system that we, where we let the particles leave little traces, and and so they they leave their trace, and then they finish their life, and then they are born again. And so literally, that's that's what's happening with those lines that you see moving.
Michael Littman: So it is a somewhat complex computation that you're doing to aggregate these readings? Right
Fernanda Viégas: it is it? But but no Noah does that.
Michael Littman: No one does that for us, and it's funny because it doesn't happen very often; but every once in a while you will see
Fernanda Viégas: that the readings from a State are a little off on the border, and so like. For instance, you will see Texas like the wind in Texas, looks different. It's not because the wind is really different. It's just like the sensors and the and the bringing together the weaving of these things. It's a little off,
I see.
Michael Littman: All right. That's yeah, that's super interesting. Um,
Michael Littman: yeah, data, you know data can be noisy, and and it's uh dirty. Is that the right where you want us to clean that? Yes, all right. That's yeah, that's really interesting. So um, let's see this is a long question, so i'm not sure what to pull out of it. Well, let me ask a different question while I read this, which is related to this question of aggregating the data and then making it available the way that you were showing it. It not only was was animating over an entire country scale of
Michael Littman: area. But then you could zoom in, and it was. It was updating, as you were, as you were doing that. How much
Michael Littman: you see a lot of your research is concerned with just the visual display of the information thinking about what to display, how to display. What colors to use, what smoothness to use It's It's beautiful stuff. It's definitely like considered.
Michael Littman: But then, to make all this happen in real time, I would think there'd be some fairly sophisticated algorithm design. Are you? Are you leveraging other People's work to do that? Or is that also part of what you're responsible for, because you know the data in new ways.
Fernanda Viégas: So I mean as much as we can reuse existing things. We will happily reuse them. But for the windmap. Specifically we did everything from scratch, because
Fernanda Viégas: the the tricky thing with the win. So,
Fernanda Viégas: even though in retrospect, the when the way we decided to visualize, when may seem very obvious. We were stuck in this project for like two months, and we almost gave up because we we have a whole talk, just talking about the failures around the women.
Fernanda Viégas: And so one of the tricky things about this kind of visualization is like you also have visualizations of fluid dynamics, and that that kind of look like like the wind up. But it's those are three, d They're much more sophisticated than what we're doing. We just needed something really to be two D,
Fernanda Viégas: and to kind of leave these traces we're talking about. So we had we. We did it all uh from scratch and and yes, to your point. We had to, you know, to make sure things were performant. We had to do things like recycling the number of particles and not drawing anything that's off screen and all of those tricks that you do. Yeah,
Michael Littman: Okay. So that's
Michael Littman: that's what it looked like. And it's it's fantastic.
Michael Littman: Just getting it right on so many of those levels is obviously a tremendous amount of work. See Maya Plankington first, she wrote, clap, Clap, because again people all over the all over the world. Today they were clapping, and you couldn't hear them. But, my! Down at the clip I figured out a clever way of getting it across. She says. Amazing work. I see this work being using criminal justice for creating better and more equitable models predicting recidivism. Is there anything about data visualization that you would like to caution people about
Michael Littman: in terms of potential misuse, especially as it pertains to understanding bias. So the more that people use this stuff the more it could potentially mislead people, either accidentally or sometimes even on How can you caution us?
Fernanda Viégas: Yes, so I um. And this is a little bit of the note that I was talking about with different voices being brought up,
Fernanda Viégas: I think, always having in the back of your mind that data. Visualization is not a neutral just because you're looking at data. It doesn't mean It's neutral in any way,
Fernanda Viégas: and it is used to persuade. In fact, it is used all the time to persuade. So if people are presenting data visualizations to you, I mean sometimes you're just like you have insights, and it's wonderful, and that's what I look forward to,
Fernanda Viégas: but also be skeptical. Be skeptical and ask questions, ask questions about the data about the framing of the data um about what scales people are using, and and again like this is going to depend a lot on what technique is being presented to you. We know how to do these things at this point much much more effectively for simple wraps. So, for instance, one of the things that's very hard for people to understand, for
Fernanda Viégas: for lay users to understand are things that differences in scale. You have a linear scale, then you have a log scale. People have no idea how to so be mindful of that.
Fernanda Viégas: At the same time, I feel like hopefully, data visualization
Fernanda Viégas: gives people more of a more confidence to ask questions about the data and about what it is that they are seeing. I really hope that this is one of the things that we get from from data visualization. So I think I think you know It's going to depend on the on the data you're looking at on the on the visualization technique. But be on the mindset of being curious and and asking questions to the person who created the visualization.
Michael Littman: Okay? Well, that is fantastic advice. And i'm going to I'm going to answer the last question myself. Well, there's okay, just to be clear. There's been a lot of other questions that I skipped. I apologize to people as questions I skipped hopefully. You still got a lot out of the answers that that we got to talk about, and the last question is, will this recording be available. So um! Ah! Blaine had actually posted that, and I will be posted right now. The link that will become live is not live yet. It will have a copy of the recording, so you can find.
Michael Littman: Just talk there. And
Michael Littman: and yeah, fantastic. So this is. It's been great getting a chance to talk to you. The Nsf. People we're going to. We're going to grab another hour of Fernando's time at three o'clock today, so we're going to have a more informal chat just amongst
Michael Littman: us folks. So please please join us for that. And I Yeah, my last question for you. Of course Fernanda is. Will Brazil win the World Cup,
Fernanda Viégas: and
Fernanda Viégas: I am who
Michael Littman: but I have to be very humble after that final game we had with Germany many years ago.
Michael Littman: I am humbly, hoping so.
Michael Littman: The best on that. All right, I think that that's it. Believe anything we need to do to close this down. I think that's it
Michael Littman: all right. Well, thank you, everybody for coming, and they're starting to disappear. Fernando a little bit really just delightful.
Fernanda Viégas: Ah, thank you so much. All right. I'll see you later.